updated PG simple example
This commit is contained in:
NT committed
1 parent
e5b3776c3b
commit
10d2815754
2 files changed
+40
-52
No files matched your search
@@ -16,6 +16,7 @@ fileList = [
|
||||
"diffphys-code-burgers.ipynb", "diffphys-code-ns.ipynb", "diffphys-code-sol.ipynb", "physicalloss-code.ipynb", # TF
|
||||
"bayesian-code.ipynb", "supervised-airfoils.ipynb", # pytorch
|
||||
"reinflearn-code.ipynb", # phiflow
|
||||
"physgrad-comparison.ipynb", # jax
|
||||
]
|
||||
|
||||
#fileList = [ "diffphys-code-burgers.ipynb"] # debug, only 1 file
|
||||
@@ -53,6 +54,7 @@ for fnOut in fileList:
|
||||
res.append( re.compile(r"InsecureRequestWarning") ) # for https download
|
||||
res.append( re.compile(r"Building wheel") ) # phiflow install, also gives weird unicode characters
|
||||
res.append( re.compile(r"warnings.warn") ) # phiflow warnings
|
||||
res.append( re.compile(r"WARNING:absl") ) # jax warnings
|
||||
# remove all "warnings.warn" from phiflow?
|
||||
|
||||
# shorten data line: "0.008612174447657694, 0.02584669669548606, 0.043136357266407785"
|
||||
|
||||
+38
-52
@@ -67,7 +67,7 @@
|
||||
"source": [
|
||||
"## Implementation\n",
|
||||
"\n",
|
||||
"For this example we'll use the [JAX framework](https://github.com/google/jax), which represents a nice alternative to pytorch and tensorflow for efficiently working with differentiable functions.\n",
|
||||
"For this example we'll use the [JAX framework](https://github.com/google/jax), which represents a nice alternative for efficiently working with differentiable functions.\n",
|
||||
"JAX also has a nice numpy wrapper that implements most of numpy's functions. Below we'll use this wrapper as `np`, and the _original_ numpy as `onp`.\n",
|
||||
"\n"
|
||||
]
|
||||
@@ -98,9 +98,7 @@
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n"
|
||||
]
|
||||
"text": []
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
@@ -284,7 +282,7 @@
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Text(0, 0.5, 'x1')"
|
||||
"<matplotlib.legend.Legend at 0x7fe6036420a0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
@@ -293,7 +291,7 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAFtCAYAAADrr7rKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAcCElEQVR4nO3df5BdZZ3n8fc3IQQ3STfWqAlCQjJSqyjU8KvUsA6ibFZFgrIrq+U4RdadXXBMRV1X1yzo6CxMZrQWcZaMUOgo7Ew5/rOFxF8I46C1ElGj/FojKys4oCSMP+hud03MkO/+cW7r5ebe23277+1zbz/vV9Wp5jz3Ofd8c7x++vRznntOZCaSpDIsqbsASdLCMfQlqSCGviQVxNCXpIIY+pJUEENfkgpi6EtSQQx9SSrIUXUXsNAiIoBnA1N11yJJfbQK+HHO8I3b4kKfKvAfrbsISRqAE4AfdetQYuhPATzyyCOMjY3VXYskzdvk5CRr166FWYxglBj6AIyNjRn6korjhVxJKoihL0kFMfQlqSCGviQVxNCXpIIY+pI0RAb9MMNaQz8i3hIR90bEZGPZHRGvmmGbiyPiexFxICLui4jzF6peSRqEqSnYtg02bIC1a6uf27ZV7f0WdT4jNyI2A08C3wcCuAR4F3B6Zv6vNv3PBr4KbAc+C7wR+E/AGZl5/yz3OQZMTExMOE9fUu2mpmDjRti7Fw4f/k37kiVw8smwezesWtX9PSYnJxkfHwcYz8zJbn1rPdPPzF2Z+fnM/H5m/u/MvBz4BfDiDpu8DfhiZn4oM/dm5nuBbwNbF6pmSeqnyy8/MvChWt+7F664or/7G5ox/YhYGhFvAFYAuzt02wjc3tJ2a6O90/suj4ix6YXqpkSSNBR27Toy8KcdPgy33NLf/dUe+hFxakT8AjgIXAdclJnf7dB9DbC/pW1/o72T7cBE0+LN1iQNhUw4dKh7n0OH+ntxt/bQBx4ATgNeBHwUuDEint/H998BjDctJ/TxvSVpziJg2bLufZYtq/r1S+2hn5m/yswHM3NPZm4H7qEau29nH7C6pW11o73T+x/MzMnpBe+jL2mIbN5cXbRtZ8kSuPDC/u6v9tBvYwmwvMNru4HzWto20fkagCQNtauuqmbptAb/9OydK6/s7/7qnqe/IyLOiYj1jbH9HcC5wF83Xr+p0TbtI8ArI+KdEfG8iHg/cBZw7ULXLkn9sGpVNS1z61ZYvx6OP776uXXr7KZr9qruefofpzpzP47qIuu9wJ9l5m2N1+8AHs7MLU3bXAxcCaynmt//7sz8fA/7dJ6+pKGV2fsYfi/z9GsN/ToY+pIWm5H5cpYkaWEZ+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqSVBBDX5IKYuhLUkEMfUkqiKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JKoihL0kFMfQlqSCGviQVxNCXpIIY+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqSVBBDX5IKUmvoR8T2iPhmRExFxOMRcXNEPHeGbbZERLYsBxaqZkkaZXWf6b8U2Am8GNgELAO+FBErZthuEjiuaTlxkEVK0mJxVJ07z8xXNq9HxBbgceBM4KvdN819AyxNkhalus/0W403fv5shn4rI+KHEfFIRHwmIl7QqWNELI+IsekFWNW3aiVpxAxN6EfEEuAa4GuZeX+Xrg8AbwZeA7yJ6t9wZ0Sc0KH/dmCiaXm0XzVL0qiJzKy7BgAi4qPAq4CXZOasgzkilgF7gU9l5nvbvL4cWN7UtAp4dGJigrGxsXlWLUn1m5ycZHx8HGA8Mye79a11TH9aRFwLXACc00vgA2TmoYj4DnBSh9cPAgeb9jWfUiVppNU9ZTMagX8R8PLMfGgO77EUOBV4rN/1SdJiU/eZ/k7gjVTj81MRsabRPpGZvwSIiJuAH2Xm9sb6+4CvAw8CxwLvopqy+bGFLV2SRk/dof+Wxs87Wtr/DfDJxn+vAw43vfZ04AZgDfBzYA9wdmZ+d2BVStIiMTQXchdKY9rmhBdyJS0WvVzIHZopm5KkwTP0Jakghr4kFcTQl6SCGPqSVBBDX5IKYuhLUkEMfUkqiKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JKoihL0kFMfQlqSCGviQVxNCXpIIY+pJUEENfkgpi6EtSQQx9SSqIoS9JBTH0Jakghr4kFcTQl6SCGPqSVBBDX5IKYuhLUkEMfUkqiKEvSQUx9CWpIIa+JBWk1tCPiO0R8c2ImIqIxyPi5oh47iy2uzgivhcRByLivog4fyHqlaRRV/eZ/kuBncCLgU3AMuBLEbGi0wYRcTbwKeDjwOnAzcDNEXHKwKuVpBEXmVl3Db8WEc8EHgdemplf7dDn08CKzLygqe3rwN2Zedks9jEGTExMTDA2NtanyiWpPpOTk4yPjwOMZ+Zkt751n+m3Gm/8/FmXPhuB21vabm20HyEilkfE2PQCrJp/mZI0moYm9CNiCXAN8LXMvL9L1zXA/pa2/Y32drYDE03Lo/OrVJJG19CEPtXY/inAG/r8vjuo/oKYXk7o8/tL0sg4qu4CACLiWuAC4JzMnOlMfB+wuqVtdaP9CJl5EDjYtK95VCpJ/ZMJCx1JdU/ZjEbgXwS8PDMfmsVmu4HzWto2NdolaahNTcG2bbBhA6xdW/3ctq1qXwi1zt6JiL8A3gi8Bnig6aWJzPxlo89NwI8yc3tj/WzgK8B7gM9RDQf9Z+CMGa4FTO/T2TuSajE1BRs3wt69cPjwb9qXLIGTT4bdu2HVHKaajNLsnbdQjbPfATzWtLy+qc864Ljplcy8k+oXxb8H7gFeB7x2NoEvSXW6/PIjAx+q9b174YorBl/DUM3TXwie6Uuqy4YN8PDDnV9fvx4ems0gd4tROtOXpCJkwqFD3fscOlT1GyRDX5IWQAQsW9a9z7Jlg5/NY+hL0gLZvLm6aNvOkiVw4YWDr8HQl6QFctVV1Syd1uCfnr1z5ZWDr8HQl6QFsmpVNS1z69bqou3xx1c/t26d+3TNXjl7R5Jq0q9v5Dp7R5JGQB13hTH0Jakghr4kFcTQl6SCGPqSVBBDX5IKYuhLUkEMfUkqiKEvSQUx9CWpIIa+JBXE0Jekghj6klQQQ1+SCmLoS1JBDH1JGrBhemyJoS9JAzA1Bdu2wYYNsHZt9XPbtqq9Tj45S5L6bGoKNm6EvXvh8OHftE8/C7ffj0b0yVmSVKPLLz8y8KFa37sXrriinrrA0Jekvtu168jAn3b4MNxyy8LW08zQl6Q+yoRDh7r3OXSovou7hr4k9VEELFvWvc+yZfU8FB0MfUnqu82bq4u27SxZAhdeuLD1PGX/9e1akhanq66qZum0Bv/07J0rr6ynLjD0JanvVq2qpmVu3Qrr18Pxx1c/t27t/3TNXjlPX5IGLHOwY/jO05ekIVLXRdt2DH1JKkjfQj8iTo6IH/S4zTkRsSsifhwRGRGvnaH/uY1+rcuaeRUvSYXo55n+0cCJPW6zArgHeGuP2z0XOK5pebzH7SWpSEfNtmNEXD1Dl2f2uvPM/ALwhcb797Lp45n5RK/7k6TSzTr0gbcBdwOdrgyvnHc1s3d3RCwH7gfen5lfW8B9S1JHg56pM1+9hP6DwIcz86/avRgRpwF7+lFUF48BlwHfApYDfwDcEREvysxvd6hreaPvtBpnyEpajKamqjtr7tpV3Vdn2bLqW7lXXVXvnPx2egn9bwFnAm1DH0hgoL/fMvMB4IGmpjsj4jnAO4Df77DZduCPBlmXpHJ1unf+zp3w5S/X/2WsVr1cyH0ncE2nFzPznsysYwroN4CTury+AxhvWk5YiKIklWGY753fzqxDOjP3ZeYPI+JlnfpExKX9Kasnp1EN+7SVmQczc3J6AWp+WJmkxWSY753fzlzOzL8YER+KiF/fPDQinhERu4A/7eWNImJlRJzWuB4AsKGxvq7x+o6IuKmp/9sj4jURcVJEnBIR1wAvB3bO4d8hSfMy7PfOb2cuof8y4CLgmxHx/Ih4NdUsmjGqs+5enAV8p7EAXN347z9urB8HrGvqfzTwX4H7gK8AvwP888z8297/GZI0P8N+7/x2ermQC0Bm3tk4M78O+DbVL473Ah/MHu/elpl30OXib2ZuaVn/IPDB3iqWpMHZvLm6aNtuiKfue+e3M9cLr/+U6iz9UeAfqb4h+0/6VZQkjYphvnd+Oz2HfkS8B9gN3AacArwQOB24NyI29rc8SRpuw3zv/HZ6vp9+RDwGvLlxC4XptmXAnwDbMnN5x42HgPfTlzRIdXwjt5f76fc8pg+cmpk/aW7IzEPAuyLis3N4P0laNIbpom07PQ/vtAZ+y2tfmV85kjQahmkaZi98iIokzdLUFGzbBhs2wNq11c9t26r2UeEzciVpFjrdY2d6lk6dF219Rq4k9dmo3WOnE0NfkmZh1O6x04mhL0kzGMV77HRi6EvSDEbxHjudGPqSNAubNx95q4Vpw3iPnU4MfUmaQebo3WOnk7l8I1eSFr12z719xSvgd38XvvjF37RdeGEV+MN2j51ODH1JatFpTv4NN1Rn9ffeCytXjsYYfiuHdySpxWzm5I9i4IOhL0lHWCxz8tsx9CWpyWKak9+OoS9JTRbTnPx2DH1JapK5eObkt+PsHUnFa52euXQpHHss/PznTx3GGbU5+e0Y+pKK1u2WyU9/ejU188knR3NOfjuGvqSidZue+cQT8KY3wTXXjO4YfivH9CUVbTbTMxdL4IOhL6lgi316ZjuGvqSiLebpme0Y+pKK0vpw85/+tHPfUZ+e2Y4XciUVo9NMnXYWw/TMdgx9ScXoNFMHqiGclSthbGzxTM9sx9CXVIxuM3Uy4bd+C37wg8U1ht/KMX1JRTh8eHYzdRY7Q1/SotV80XbdOtiLine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAFtCAYAAADrr7rKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAfu0lEQVR4nO3df5RcZZ3n8fe3O51Aku6OI5IESEyYzCIMnAFhHYIzCGJWxzGAu6BzRufIOrOCkBPGdf2RRV3HIWTWOatklogcnRnJOsfhzDl7gPiLH8OgZyWiREAZY3ZYA5NgQhCluxXzA/rZP241FJWq7q5OVd+qft6vc+5p6tZzq75cik/ffu637o2UEpKkPPSUXYAkafoY+pKUEUNfkjJi6EtSRgx9ScqIoS9JGTH0JSkjhr4kZWRW2QVMt4gI4DhgpOxaJKmF+oGfpAm+cZtd6FME/u6yi5CkNjgBeGK8ATmG/gjArl27GBgYKLsWSTpiw8PDLFmyBCYxg5Fj6AMwMDBg6EvKjidyJSkjhr4kZcTQl6SMZDunP56UEs899xzPP/982aV0jN7eXmbNmkXR8SqpWxn6NQ4ePMiePXt49tlnyy6l48ydO5fFixcze/bsskuRNEWGfpXR0VF27txJb28vxx13HLNnz/bIluIvn4MHD/LUU0+xc+dOfuM3foOeHmcGpXZICdoZO6WGfkS8F3gvsKyy6p+BT6SUvjbONpcCf17Z5l+AD6WUvtqKeg4ePMjo6ChLlixh7ty5rXjJGePoo4+mr6+Pxx9/nIMHD3LUUUeVXZI0Y4yMwDXXwJYtcOgQ9PXB6tWwfj3097f2vco+0t8NfJgivAN4F3BbRJyRUvrn2sERcQ7wJWAd8GXgD4FbI+LVKaVHWlWUR7H1uV+k1hsZgZUrYft2GB19cf2mTXDPPbB1a2uDv9T/i1NKW1JKX00p/UtK6f+mlK4BfgGc3WCTq4Gvp5T+MqW0PaX0UeB7wJrpqlmSWumaaw4PfCgeb98OH/lIa9+vYw7dIqI3Iv4AmAdsbTBsJXB3zbo7Kusbve6ciBgYWyguSiRJHWHLlsMDf8zoKNx+e2vfr/TQj4jTIuIXwAHgs8BbU0o/bDB8EfBkzbonK+sbWQcMVS1ebE1SR0ipmMMfz6FDxbhWKT30gR3A6cBvAzcCN0fEKS18/Q3AYNVyQgtfu6Ps3buXq6++mhUrVnDUUUexcOFCXvva13LjjTe+0IK6bNkyIoKI4Oijj2bZsmW87W1v45577im5eik/EcVJ2/H09bW2m6f00E8pHUwpPZpS2pZSWgc8TDF3X89eYGHNuoWV9Y1e/0BKaXhsYZqvo9/K39Dj+fGPf8wZZ5zBnXfeyXXXXceDDz7I1q1b+eAHP8iXv/xl7r77xVmxT3ziE+zZs4cdO3awefNmFixYwBve8AbWr18/PcVKesHq1dCoR6KnBy68sLXvV3b3Tj09wJwGz20FLgCur1q3isbnAEoxne1XY6688kpmzZrFAw88wLx5815Yf+KJJ3LRRRdRfV+F/v5+Fi0qZsSWLl3Kueeey+LFi/nYxz7GJZdcwkknndSeIiUdZv36okun9mRuTw+cfDJce21r36/UI/2I2BAR50bEssrc/gbgPODvKs9vrqwbsxF4U0S8PyJeFREfB84Cbpju2hsZa7/atAkeewyeeKL4uWlTsX6kDX9nPP3009x5551cddVVLwn8ahN9yezqq68mpcRtt93W+gIlNdTfX7RlrlkDy5bB8ccXP9esaX27JpQ/vXMssJliXv8fgX8LvDGldFfl+aXA4rHBKaX7KHrz30MxDXQJcHEre/SP1HS3XwE8+uijpJQOO0I/5phjmD9/PvPnz+dDH/rQuK/xa7/2axx77LE89thjrS9Q0rj6+2HjRti5E3btKn5u3NiemYFSp3dSSn88wfPn1Vn3D8A/tKumIzWZ9quNG6enlu985zuMjo7yjne8gwMHDkw4PqXkZSekkrX7f8FOnNPvWs20X7XyP+yKFSuICHbs2PGS9SeeeCJQXEJhIk8//TRPPfUUy5cvb11hkjpO2dM7M0oZ7VcAL3/5y1m1ahU33HADv/zlL6f0Ghs3bqSnp4eLL764tcVJ6iiGfotNd/vVmM985jM899xznHXWWdxyyy1s376dHTt28MUvfpEf/ehH9Pb2vjB2ZGSEvXv3smvXLr75zW/ynve8h2uvvZb169ezYsWK9hQoqSNEmq5G8g5RuRTD0NDQ0GE3Rt+/fz87d+5k+fLlU76KZKOLJ421X7XjbPyYPXv2cN111/GVr3yF3bt3M2fOHE455RQuvfRSrrzySubOncuyZct4/PHHAZg9ezaLFi3i7LPP5oorruD8888f9/VbsX8ktd7w8DCDg4MAg5XvIzVk6FdpVaiNjBRdOrff/mKf/oUXFv227Qr86WDoS52pmdD3RG4bjLVfbdzY/hsiSFIznNNvMwNfUicx9CUpI4a+JGXE0K8jt5Pbk+V+kbqfoV+lr/LNqrFrz+ulxvZL30TfQJPUsezeqdLb28uCBQvYt28fAHPnzvVaNBRH+M8++yz79u1jwYIFL/mil6TuYujXGLvO/Fjw60ULFix4Yf9I6k6Gfo2IYPHixRx77LEcmujqaRnp6+vzCF+aAQz9Bnp7ew05STOOJ3IlKSOGviRlxNCXpIwY+pKUEUNfkjJi6EtSRgx9ScqIoS9JGTH0JSkjhr4kZcTQl6SMGPqSlBFDX5IyYuhLUkYMfUnKiKEvSRkx9CUpI4a+JGXE0JekjBj6kpSRUkM/ItZFxHcjYiQi9kXErRFx0gTbXBYRqWbZP101S1I3K/tI/3XAJuBsYBXQB9wZEfMm2G4YWFy1vLKdRUrSTDGrzDdPKb2p+nFEXAbsA84Evjn+pmlvG0uTpBmp7CP9WoOVnz+bYNz8iHg8InZFxG0R8ZuNBkbEnIgYGFuA/pZVK0ldpmNCPyJ6gOuBb6WUHhln6A7g3cBFwDsp/h3ui4gTGoxfBwxVLbtbVbMkdZtIKZVdAwARcSPwe8DvpJQmHcwR0QdsB76UUvponefnAHOqVvUDu4eGhhgYGDjCqiWpfMPDwwwODgIMppSGxxtb6pz+mIi4AXgLcG4zgQ+QUjoUEQ8CKxo8fwA4UPVeR1KqJHW1sls2oxL4bwVen1LaOYXX6AVOA/a0uj5JmmnKPtLfBPwhxfz8SEQsqqwfSin9CiAiNgNPpJTWVR5/DPg28CiwAPgARcvm56e3dEnqPmWH/nsrP++tWf8fgS9U/nkpMFr13MuAzwGLgJ8D24BzUko/bFuVkjRDdMyJ3OlSadsc8kSupJmimRO5HdOyKUlqP0NfkjJi6EtSRgx9ScqIoS9JGTH0JSkjhr4kZcTQl6SMGPqSlBFDX5IyYuhLUkYMfUnKiKEvSRkx9CUpI4a+JGXE0JekjBj6kpQRQ1+SMmLoS1JGDH1JyoihL0kZMfQlKSOGviRlxNCXpIwY+pKUEUNfkjJi6EtSRgx9ScqIoS9JGTH0JSkjhr4kZcTQl6SMGPqSlBFDX5IyYuhLUkZKDf2IWBcR342IkYjYFxG3RsRJk9ju0oj4UUTsj4gfRMSbp6NeSep2ZR/pvw7YBJwNrAL6gDsjYl6jDSLiHOBLwF8DZwC3ArdGxKltr1aSulyklMqu4QUR8QpgH/C6lNI3G4y5BZiXUnpL1bpvAw+llK6YxHsMAENDQ0MMDAy0qHJJKs/w8DCDg4MAgyml4fHGln2kX2uw8vNn44xZCdxds+6OyvrDRMSciBgYW4D+Iy9TkrpTx4R+RPQA1wPfSik9Ms7QRcCTNeuerKyvZx0wVLXsPrJKJal7dUzoU8ztnwr8QYtfdwPFXxBjywktfn1J6hqzyi4AICJuAN4CnJtSmuhIfC+wsGbdwsr6w6SUDgAHqt7rCCqVpNZJCaY7kspu2YxK4L8VeH1KaeckNtsKXFCzblVlvSR1tJERWLsWli+HJUuKn2vXFuunQ6ndOxHxGeAPgYuAHVVPDaWUflUZsxl4IqW0rvL4HOAbwIeBr1BMB/1X4NUTnAsYe0+7dySVYmQEVq6E7dthdPTF9T09cPLJsHUr9E+h1aSbunfeSzHPfi+wp2p5e9WYpcDisQcppfsoflG8B3gYuAS4eDKBL0lluuaawwMfisfbt8NHPtL+GjqqT386eKQvqSzLl8NjjzV+ftky2DmZSe4a3XSkL0lZSAkOHRp/zKFDxbh2MvQlaRpEQF/f+GP6+trfzWPoS9I0Wb26OGlbT08PXHhh+2sw9CVpmqxfX3Tp1Ab/WPfOtde2vwZDX5KmSX9/0Za5Zk1x0vb444ufa9ZMvV2zWXbvSFJJWvWNXLt3JKkLlHFVGENfkjJi6EtSRgx9ScqIoS9JGTH0JSkjhr4kZcTQl6SMGPqSlBFDX5IyYuhLUkYMfUnKiKEvSRkx9CUpI4a+JGXE0JekNuuk25YY+pLUBiMjsHYtLF8OS5YUP9euLdaXyTtnSVKLjYzAypWwfTuMjr64fuxeuK2+NaJ3zpKkEl1zzeGBD8Xj7dvhIx8ppy4w9CWp5bZsOTzwx4yOwu23T2891Qx9SWqhlODQofHHHDpU3sldQ1+SWigC+vrGH9PXV85N0cHQl6SWW726OGlbT08PXHjh9Nbzkvcv760laWZav77o0qkN/rHunWuvLacuMPQlqeX6+4u2zDVrYNkyOP744ueaNa1v12yWffqS1GYptXcO3z59SeogZZ20rcfQl6SMtCz0I+LkiPhxk9ucGxFbIuInEZEi4uIJxp9XGVe7LDqi4iUpE6080p8NvLLJbeYBDwNXNbndScDiqmVfk9tLUpZmTXZgRHxqgiGvaPbNU0pfA75Wef1mNt2XUnqm2feTpNxNOvSBq4GHgEZnhucfcTWT91BEzAEeAT6eUvrWNL63JDXU7k6dI9VM6D8KfDql9MV6T0bE6cC2VhQ1jj3AFcADwBzgT4BLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 400x400 with 1 Axes>"
|
||||
]
|
||||
@@ -309,10 +307,9 @@
|
||||
"axes = plt.figure(figsize=(4, 4), dpi=100).gca()\n",
|
||||
"historyGD = onp.asarray(historyGD)\n",
|
||||
"updatesGD = onp.asarray(updatesGD) # for later\n",
|
||||
"axes.scatter(historyGD[:,0], historyGD[:,1], lw=0.5, color='blue')\n",
|
||||
"axes.scatter(historyGD[:,0], historyGD[:,1], lw=0.5, color='blue', label='GD')\n",
|
||||
"axes.scatter([0], [0], lw=0.25, color='black', marker='x') # target at 0,0\n",
|
||||
"axes.set_xlabel('x0')\n",
|
||||
"axes.set_ylabel('x1')"
|
||||
"axes.set_xlabel('x0'); axes.set_ylabel('x1'); axes.legend()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -406,7 +403,7 @@
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Text(0, 0.5, 'x1')"
|
||||
"<matplotlib.legend.Legend at 0x7fe6025aab80>"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
@@ -415,7 +412,7 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAFtCAYAAADrr7rKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAgpklEQVR4nO3df5RkZX3n8fe3h2Fgh+7GrGbAAZwJHAwGMiicxGE9iJJJFB3A3bB6jIkTNyu69hnjumadhSSaZUJ+nKhknSirRiHZY/xnF6ejBjAEcyIj0VEmso4QDOgwMkMU6G6zztjQ3/3jVmvRXdXd1VNV91bf9+uce4q69dyq7xQ1n7l1n6eeJzITSVI9DJVdgCSpfwx9SaoRQ1+SasTQl6QaMfQlqUYMfUmqEUNfkmrE0JekGjmu7AL6LSICeDYwVXYtktRFw8C3c5Ff3NYu9CkC/+Gyi5CkHjgNOLhQgzqG/hTAgQMHGBkZKbsWSTpmk5OTnH766bCEKxh1DH0ARkZGDH1JtWNHriTViKEvSTVi6EtSjRj6klQjhr4k1YihL0kVkjO9Xc2w1NCPiDdHxD9ExGRj2xMRL1/kmKsi4usRcSQivhoRl/WrXknqhanHp7jzj7ZzYNdGDt14Ogd2beTOP9rO1OPdnzggylwjNyK2Ak8B/wgE8HrgHcDzM/P/tmh/EfC3wA7gL4HXAv8VeEFm3rvE1xwBJiYmJhynL6l0U49PcejmzfzEj+1n1aqZH+5/8qkhHnzsHE75lT0MP2N4weeYnJxkdHQUYDQzJxdqW+qZfmaOZ+anM/MfM/P+zLwG+B7wwjaHvBX4q8z8w8zcn5m/CXwZGOtXzZLUTXv/9Jp5gQ9w3KoZNv7YfvZ+9Nquvl5lrulHxKqIeA2wFtjTptlm4LNz9t3a2N/ueddExMjsRjEpkSRVwpknjM8L/FnHrZrhzON3d/X1Sg/9iDgvIr4HHAU+CLwqM7/WpvkpwOE5+w439rezA5ho2pxsTVIl5Exy3ND0gm2OG5ruaudu6aEP3AecD/ws8AHgpoh4Xhef/3pgtGk7rYvPLUnLFkPBkzOrF2zzZK4mhqJrr1l66GfmDzLzgczcm5k7gH0U1+5bOQSsm7NvXWN/u+c/mpmTsxvOoy+pQr5xZCtPPtU6ip98aohvHL28q69Xeui3MASsafPYHuDSOfu20L4PQJIq7YI37OTBx86ZF/xPPjXEg989hwt+9bquvl6pUytHxPXAZ4BvUXSwvha4BPiFxuM3Awcb3wAAbgA+FxFvBz4FvAa4EHhjfyuXpO4YfsYw/Moe/u6j13Lm8bs5bmiaJ2dW840fXM4Fv3rdosM1O1X2OP2PUJy5n0rRyfoPwO9n5u2Nx+8EHsrMbU3HXAVcB2ygGN//G5n56Q5e03H6kiorZ7Lja/idjNMvNfTLYOhLWmkG5sdZkqT+MvQlqUYMfUmqEUNfkmrE0JekGjH0JalGDH1JqhFDX5JqxNCXpBox9CWpRgx9SaoRQ1+SasTQl6QaMfQlqUYMfUmqEUNfkmrE0JekGjH0JalGDH1JqhFDX1JrNVs/uy6OK7sASRUyPQX7roGD4zAzDUOrYf1W2LQTVg+XXZ26wNCXVJiegts2w8R+YOZH++/fBYfvgJ/fY/CvAF7ekVTYd838wIfi/sR+2HdtGVWpywx9SYWD48wP/FkzcHB3P6tRjxj6kopO25nphdvMTNu5uwIY+pIgoui0XcjQ6qKdBpqhL6mwfivtI2EI1l/ez2rUI4a+pMKmnTB6DvNjYajYv+m6MqpSlxn6kgqrh4thmWePwdoNcOL64vbsMYdrriCRNeuYiYgRYGJiYoKRkZGyy5GqK9Nr+ANicnKS0dFRgNHMnFyorWf6kloz8FckQ1+SasTQl6QaMfQlqUYMfUmqkVJDPyJ2RMQXI2IqIh6NiFsi4rmLHLMtInLOdqRfNUvSICv7TP/FwC7ghcAWYDVwW0SsXeS4SeDUpu05vSxSklaKUufTz8yXNd+PiG3Ao8AFwN8ufGge6mFpkrQilX2mP9do4/axRdqdFBHfjIgDEfHJiPipdg0jYk1EjMxugD8rlFRblQn9iBgC3gd8PjPvXaDpfcAbgCuA11H8Ge6KiNPatN8BTDRtD3erZkkaNJWZhiEiPgC8HHhRZi45mCNiNbAf+Hhm/maLx9cAa5p2DQMPOw2DpJWik2kYKrFGbkS8H3glcHEngQ+QmdMR8RXgrDaPHwWONr3WsZQqSQOt7CGb0Qj8VwEvzcwHl/Ecq4DzgEe6XZ8krTRln+nvAl5LcX1+KiJOaeyfyMzvA0TEzcDBzNzRuP9bwBeAB4CTgXdQDNn8cH9Ll6TBU3bov7lxe+ec/b8KfKzx32fw9NWanwF8CDgFeBzYC1yUmV/rWZWStEJUpiO3X5xPX9JK43z6UjfV7MRIK1vZl3ekapqegn3XwMFxmJmGodXFwuGbdrpsoAaaoS/NNT0Ft22Gif08rTvp/l1w+A7Xi9VA8/KONNe+a+YHPhT3J/bDvmvLqErqCkNfmuvgOPMDf9YMHNzdz2qkrjL0pWaZxTX8hcxM27mrgWXoS80iik7bhQytLtpJA8jQl+Zav5X2fzWGYP3l/axG6ipDX5pr004YPYf5fz2Giv2briujKqkrDH1prtXDxbDMs8dg7QY4cX1xe/aYwzU18JyGQVpMptfwVWlOwyB1k4GvFcTQl6QaMfQlqUYMfUmqEUNfkmrE0JekGjH0JalGDH1JqhFDX5JqxNCXpBox9CWpRgx9SaoRQ1+SasTQV3lqNsOrVAXHlV2AamZ6CvZdUyw+PjNdLD24fmuxcInz1Es9Z+irf6an4LbNMLEfmPnR/vt3weE7XKBE6gMv76h/9l0zP/ChuD+xH/ZdW0ZVUq0Y+uqfg+PMD/xZM3Bwdz+rkWrJ0Fd/ZBbX8BcyM23nrtRjhr76I6LotF3I0GqXJpR6zNBX/6zfSvuP3BCsv7yf1Ui1ZOirfzbthNFzmP+xGyr2b7qujKqkWjH01T+rh4thmWePwdoNcOL64vbsMYdrSn0SWbOOs4gYASYmJiYYGRkpu5x6y/QavtQFk5OTjI6OAoxm5uRCbUs904+IHRHxxYiYiohHI+KWiHjuEo67KiK+HhFHIuKrEXFZP+pVlxn4Ut+VfXnnxcAu4IXAFmA1cFtErG13QERcBHwc+AjwfOAW4JaIOLfn1UrSgKvU5Z2IeBbwKPDizPzbNm0+AazNzFc27fsCcE9mvmkJr+HlHUkrysBc3mlhtHH72AJtNgOfnbPv1sb+eSJiTUSMzG6AvYWSaqsyoR8RQ8D7gM9n5r0LND0FODxn3+HG/lZ2ABNN28PHVqkkDa7KhD7Ftf1zgdd0+Xmvp/gGMbud1uXnl6SBUYmplSPi/cArgYszc7Ez8UPAujn71jX2z5OZR4GjTa91DJVKUveUMWq57CGb0Qj8VwEvzcwHl3DYHuDSOfu2NPZLUqVNTcH27bBxI5x+enG7fXuxvx9KHb0TEX8CvBa4Ariv6aGJzPx+o83NwMHM3NG4fxHwOeCdwKcoLgf9N+AFi/QFzL6mo3cklWJqCjZvhv37YaZplvGhITjnHNizB4aXMdRkkEbvvJniOvudwCNN26ub2pwBnDp7JzPvoviH4o3APuAXgSuXEviSVKZrrpkf+FDc378fru3DOkKVGqffD57pSyrLxo3w0EPtH9+wAR5cykXuOQbpTF9VUrMTAKmfMmF6kXWEpvuwjlAlRu+oRNNTxdq1B8eLlauGVhfz3m/a6ayXUhdFwOpF1hFa3Yd1hDzTr7PpKbhtM9y/C/7lIfj+weL2/l3F/uk+DSeQamLr1qLTtpWhIbi8D+sIGfp1tu8amNjP/MXKZ4r9+/rQqyTVyM6dxSiducE/O3rnuj6sI2To19nBceYH/qwZOLi7n9VIK97wcDEsc2ys6LRdv764HRtb/nDNTnlNv64yi2v4C5mZdqETqcuGh+GGG4qtdr/IVYkiik7bhQz1oVdJqrEy/noZ+nW2fivtPwJDsL4PvUqS+srQr7NNO2H0HOZ/DIaK/Zv60Kskqa8M/TpbPQw/vwfOHoO1G+DE9cXt2WPFfsfpSyuO0zDoR+y0lQaS0zBoeQx8acUz9CWpRgx9SaoRQ1+SasTQl6QaMfQlqUYMfUmqEUNfkmrE0B90NftxnaRj49TKg8glDiUtk6E/aGaXOJy74tX9u+DwHc6ZI1VQlWY48fLOoHGJQ2kgTE3B9u2wcSOcfnpxu317sb9MTrg2aD65sVi8vJ21G+CKB/tVjaQWpqZg82bYvx9mms7PZtfC7fbSiE64tlJ1ssShpNJcc838wIfi/v79cG2JX8gN/UHiEofSQBgfnx/4s2ZmYPfu/tbTzNAfNC5xKFVaJkwv8oV8usQv5Ib+oHGJQ6nSImD1Il/IV5f4hdzQHzQucShV3tatRadtK0NDcHmJX8gdvTPoqjQAWBLg6B31koEvVc7wcBHsY2OwYQOsX1/cjo11P/A75Zm+JPVYr7+Qe6YvSRVSpS/khr4k1UjXQj8izomIf+rwmIsjYjwivh0RGRFXLtL+kka7udspx1S8JNVEN8/0jwee0+Exa4F9wFs6PO65wKlN26MdHi9JtbTkqZUj4j2LNHlWpy+emZ8BPtN4/k4OfTQzn+j09QaCQzAl9VAn8+m/FbgHaNczfNIxV7N090TEGuBe4F2Z+fk+vnb3uSiKtGJU/bytk9B/AHhvZv55qwcj4nxgbzeKWsAjwJuALwFrgF8D7oyIn83ML7epa02j7axqpaiLokgDb2qqmFlzfLyYV2f16uJXuTt3ljsmv5VOQv9LwAVAy9AHEujpv2+ZeR9wX9OuuyLiTOBtwC+3OWwH8Nu9rOuYLGVRlAtvKKMySUvQ7te3u3bBHXeU/2OsuTrpyH078L52D2bmvswsYwjo3wNnLfD49cBo03ZaP4pasoPjzA/8WTNwsMQ5WCULine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAFtCAYAAADrr7rKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAo40lEQVR4nO3df3RU9Z3/8ec7IfxOglYliNCgVItFAWW3ggpYZXXbIrpW6altZbv7rVZz0K7f/uCLa9UV0d3zbcVC1W+7LdJuXU7P+X6B2Hb9uaSrUn+gUF0i/gILlB/+IonVhMi8v3/cSRwmM0kmmZl7J/f1OOeeMHc+d+bNMLxy53M/8/mYuyMiIvFQFnYBIiJSPAp9EZEYUeiLiMSIQl9EJEYU+iIiMaLQFxGJEYW+iEiMKPRFRGJkUNgFFJuZGXAs0BJ2LSIieVQJ/Ml7+MZt7EKfIPB3hV2EiEgBHAfs7q5BHEO/BWDnzp1UVVWFXYuISL81Nzczbtw46EUPRhxDH4CqqiqFvojEji7kiojEiEJfRCRGFPoiIjES2z797rg7H374IYcOHQq7FAHKy8sZNGgQwWhbEekPhX6agwcPsmfPHt5///2wS5EUw4cPZ8yYMQwePDjsUkRKmkI/RSKRYPv27ZSXl3PssccyePBgnV2GzN05ePAgb775Jtu3b+cTn/gEZWXqlZSByxOOlRUud0INfTP7BvANoDa567+BW9z9t90ccynwT8ljXgG+4+6/yUc9Bw8eJJFIMG7cOIYPH56Ph5Q8GDZsGBUVFbzxxhscPHiQoUOHhl2SSF61vNvCpp8u4YSh9Qwqa+fDRAWvtc7j9K8tpfKIyrw+V9hn+ruA7xKEtwFXAOvMbJq7/3d6YzObCdwPLAYeAL4ErDWz09z9xXwVpTPJ6NG/iQxULe+2sHf1DM4+ppHy8kTn/jGHVrJ99WPw1Y15Df5Q/ye5e727/8bdX3H3l919CfAecEaWQ64F/sPd/8XdG939H4HngLpi1Swikk+bfrqE4488PPABBpUnmHBkI5t+dkNeny8yp09mVm5mXwRGABuzNJsBPJK278Hk/myPO8TMqjo2gkmJREQi4YSh9V0Cv8Og8gQnDF6f1+cLPfTN7BQzew9oA+4BLnb3rVma1wD70vbtS+7PZjHQlLJpsjURiQRPOIPK2rttM6isHU90O3FmTkIPfWAbMBX4NHA3cJ+ZnZzHx18GVKdsx+XxsSNl7969XHvttUycOJGhQ4cyevRozjzzTO6+++7OIai1tbWYGWbGsGHDqK2t5bLLLuOxxx4LuXqR+LEy48NERbdtPvSKvI7mCT303f2gu7/q7pvcfTGwhaDvPpO9wOi0faOT+7M9fpu7N3dsFHke/e5nts6f119/nWnTpvHQQw9x22238fzzz7Nx40a+/e1v88ADD/DIIx/1it1yyy3s2bOHbdu2sXr1akaNGsV5553H0qVLi1OsiHR6rXUeHx7KHMUfHirjtbYL8/p8YY/eyaQMGJLlvo3AucCdKfvmkv0aQChaWmDJEqivh/Z2qKiAefNg6VKoLNAVhauvvppBgwbx7LPPMmLEiM79xx9/PPPnzyd1XYXKykpqaoIesfHjxzNr1izGjBnDjTfeyBe+8AVOOumkwhQpIl2c/rWlbF/9GBOObGRQSt/+h4fK2P72JE7/21vz+nyhnumb2TIzm2Vmtcm+/WXAHODfkvevTu7rsBy4wMyuN7NPmtlNwHRgRbFrz6alBWbMgJUrYccO2L07+LlyZbC/pQCfM95++20eeughrrnmmsMCP1VPXzK79tprcXfWrVuX/wJFJKvKIyqp+epGHn+zjp3v1LLnwFh2vlPL42/WUXNFfodrQvhn+scAq4ExBBdZ/wCc7+4PJ+8fD3T+6nP3J83sS8CtwG0E4/svyucY/f5asgQaGyGRdjE+kQj233ADLF+e3+d89dVXcfcuZ+hHHXUUra2tAFxzzTXccccdWR/jyCOP5JhjjmHHjh35LU5EelR5RCVz/mE5sLzzG7njCvRcoYa+u/9dD/fPybDvV8CvClVTf9XXdw38DokErF+f/9DP5umnnyaRSHD55ZfT1tbWY3t317QTIiEr5BQMEP6Z/oDiHvThd6e9PWiXz2ydOHEiZsa2bdsO23/88ccDwTQGPXn77bd58803mTBhQv4KE5HICX30zkBiFly07U5FRX4DH+BjH/sYc+fOZcWKFfz5z3/u02MsX76csrIyLrroovwWJyKRotDPs3nzINs0MWVlcGF+R191+tGPfsSHH37I9OnTWbNmDY2NjWzbto1f/OIXvPTSS5SXl3e2bWlpYe/evezcuZPf/e53fP3rX+fWW29l6dKlTJw4sTAFikgkmBdrIHlEJKdiaGpqauqyMHprayvbt29nwoQJfZ7JsWP0TvrF3LIymDQJNm4s3LDNPXv2cNttt/HrX/+aXbt2MWTIEE4++WQuvfRSrr76aoYPH05tbS1vvPEGAIMHD6ampoYzzjiDq666inPOOacwheVBPv5tRAaq5uZmqqurAaqT30fKSqGfIl/B0tISjNJZv/6jcfoXXgi33lq4wB/oFPoi2eUS+rqQWwCVlcEIneXL83/RVkSkP9SnX2AKfBGJEoW+iEiMKPRFRGJEoS8iEiMKfRGRGFHoi4jEiEJfRCRGFPoiIjGi0B8gFi5ciJlx++23H7Z/7dq1BZsu+aabbmLq1KkFeWwRKQyF/gAydOhQ7rjjDt59992wSxGRiFLoF1oR5zY677zzqKmpYdmyZVnbPP7445x99tkMGzaMcePGsWjRos7pmFesWMHkyZM723Z8SrjnnnsOe44bbriBVatWcfPNN7NlyxbMDDNj1apVAPzxj39k/vz5jBw5kqqqKi677DL27dvX+RgdnxB+/vOfU1tbS3V1NV/84hdpKcRakiJyGIV+IbS3wLOLYN0EWDsu+PnsomB/AZWXl3Pbbbfxwx/+kF27dnW5/7XXXuOCCy7gkksu4Q9/+ANr1qzh8ccfp66uDoDZs2ezdetW3nzzTQAaGho46qij2LBhQ/DXam9n48aNzJkzhwULFnD99dfzqU99ij179rBnzx4WLFhAIpFg/vz5vPPOOzQ0NPDwww/z+uuvs2DBgi61rF27lgceeIAHHniAhoaGLl1TIpJ/Cv18a2+Bh2bAyyvhzzvgg93Bz5dXBvsLHPwXX3wxU6dO5Xvf+16X+5YtW8bll1/Oddddxyc+8QlmzpzJXXfdxerVq2ltbWXy5MkceeSRNDQ0ALBhwwauv/76zttPP/007e3tzJw5k2HDhjFy5EgGDRpETU0NNTU1DBs2jEcffZQXXniBX/7yl5x++ul8+tOfZvXq1TQ0NPDMM8901pJIJFi1ahWTJ0/m7LPP5itf+QqPPvpoQV8bEVHo59+WJdDUSMp67kmJYP+WGwpewh133MF9991HY2Pj4aVt2cKqVasYOXJk53b++eeTSCTYvn07ZsasWbPYsGEDBw4cYOvWrVx99dW0tbXx0ksv0dDQwF/8xV8wfPjwrM/d2NjIuHHjGDfuo2WdTz75ZEaNGnVYPbW1tVSmzDM9ZswY9u/fn8dXQfotZtOux4VCP99219M18DskYPf6gpcwa9Yszj//fBYvXnzY/vfee48rr7ySzZs3d25btmzhlVde4YQTTgBgzpw5bNiwgf/6r/9i2rRpVFVVdf4iaGhoYPbs2XmpsSJtXUkzI5FtRXkpnpC6JqV4NJ9+PrlDooeV0RMFWBk9g9tvv52pU6dy0kknde477bTT2Lp1a7dLIs6ePZvrrruOX/3qV8yZMwcIfhE88sgjPPHEE1x//fWdbQcPHsyhQ4cOO37SpEns3LmTnTt3dp7tb926lQMHDnDyySfn8W8oedfRNZn+SfXllbDvMfirjVChVYBKnc7088kMynpYGb2sACujZ3DKKadw+eWXc9ddd3Xu+853vsOTTz5JXV0dmzdv5pVXXmHdunWdF3IBTj31VI444gh++ctfHhb6a9eupa2tjTPPPLOzbW1tLdu3b2fz5s289dZbtLW1cd5553U+93PPPcfTTz/NV7/6VWbPns306dML/veWfohA16QUnkI/38bOI/vLWgZjC7Qyega33HLLYV0mp556Kg0NDbz88sucffbZTJs2jRtvvJFjjz22s42ZcfbZZ2NmnHXWWZ3HVVVVMX36dEaMGNHZ9pJLLuGCCy7gnHPO4eijj+b+++/HzFi3bh1HHHEEs2bN4rzzzuP4449nzZo1Rft7Sx9FoGtSCk9r5KbIyzqs2T4iUwbVk/QRuY+0Rm6BuQd9+B/szt5m2Fi4aKeWg4ugXNbI1Zl+vlVUBsF+Yh2MqA3+o4yoDW4r8CWqItQ1KYWlC7mFUFEJ05cHm1ZGl1Ixdl5w0TZjF09xuyalcHSmX2gKfCkVU5YGXZBdYiHZNTnl1jCqkjxT6ItIQF2TsaDuHRH5iLomBzyd6WcQtxFNpUD/JiFQ4A9ICv0UHVMDvP/++yFXIuk6/k3Sp28QkdyoeydFeXk5o0aN6pz4a/jw4QVbdUp6x915//332b9/P6NGjaK8vDzskkRKmkI/TU1NDYBmfIyYUaNGdf7biEjfKfTTmBljxozhmGOOob29h8nTpCgqKip0hi+SJ6GGvpktBv4G+CTwAfAk8B1339bNMQuBn6XtbnP3vH43v7y8XEEjIgNO2BdyZwMrgTOAuUAF8JCZjej2KGgGxqRsHy9kkSIiA0WoZ/rufkHq7eRZ/H7gdOB33R/qewtYmojIgBT2mX666uTPd3poN9LM3jCznWa2zsw+la2hmQ0xs6qODdDXCkUktiIT+mZWBtwJPOHuL3bTdBvwNWA+8GWCv8OTZnZclvaLgaaUbVe+ahYRKTWRmU/fzO4G/ho4y917HcxmVgE0Ave7+z9muH8IMCRlVyWwK9N8+iIipSiX+fQjMWTTzFYAnwdm5RL4AO7ebmbPAxkXfnXLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 400x400 with 1 Axes>"
|
||||
]
|
||||
@@ -430,11 +427,10 @@
|
||||
"axes = plt.figure(figsize=(4, 4), dpi=100).gca()\n",
|
||||
"historyNt = onp.asarray(historyNt)\n",
|
||||
"updatesNt = onp.asarray(updatesNt) \n",
|
||||
"axes.scatter(historyGD[:,0], historyGD[:,1], lw=0.5, color='blue')\n",
|
||||
"axes.scatter(historyNt[:,0], historyNt[:,1], lw=0.5, color='orange')\n",
|
||||
"axes.scatter(historyGD[:,0], historyGD[:,1], lw=0.5, color='blue', label='GD')\n",
|
||||
"axes.scatter(historyNt[:,0], historyNt[:,1], lw=0.5, color='orange', label='Newton')\n",
|
||||
"axes.scatter([0], [0], lw=0.25, color='black', marker='x') # target at 0,0\n",
|
||||
"axes.set_xlabel('x0')\n",
|
||||
"axes.set_ylabel('x1')"
|
||||
"axes.set_xlabel('x0'); axes.set_ylabel('x1'); axes.legend()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -531,7 +527,7 @@
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Text(0, 0.5, 'x1')"
|
||||
"<matplotlib.legend.Legend at 0x7fe601e7c070>"
|
||||
]
|
||||
},
|
||||
"execution_count": 9,
|
||||
@@ -540,7 +536,7 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAFtCAYAAADrr7rKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAi1UlEQVR4nO3df5RcZZ3n8fe3kiawSXczjppAk5ggJxgFG4SjJstBhI0iEn44MnpcZ8m6M4JLn6jrOks2jKsjMTPOWSSztJpVR8joOPwX0qMOP4aB3TUtI9E0ohFkTSRpSFCR7tYloUl/94+nCivVdauruqvqPrfu53XOPUXdem7Vk6L6U7fufe73MXdHRETyoZB2B0REpH0U+iIiOaLQFxHJEYW+iEiOKPRFRHJEoS8ikiMKfRGRHFHoi4jkyPy0O9BuZmbAqcBE2n0REWmibuApn+GK29yFPiHwD6bdCRGRFjgNGK3VII+hPwFw4MABenp60u6LiMicjY+Ps3TpUqjjCEYeQx+Anp4ehb6I5I5O5IqI5IhCX0QkRxT6IiI5otAXEckRhb6ISI4o9EVEIuLHplr6/KmGvpl9yMweMbPx4jJsZu+YYZtrzOwnZnbEzH5oZpe1q78iIq0w8eRTHLygnxd/fz5TL+/ixd+fz8EL+pl48qmmv1bae/oHgRuB84DzgfuBu8zsddUam9ka4BvAV4BzgR3ADjM7qy29FRFpsoknn+KEN51O365HmP/sMeY9N8X8Z4/RN/wIJ7zp9KYHv8U2MbqZPQt83N2/UuWxO4GF7n552brvAnvc/fo6n78HGBsbG9PFWSKSuoMX9NO36xGsShR7AUbX9HPa/95T8znGx8fp7e0F6HX38Vpt097Tf4mZzTOz9wILgeGEZquB+yrW3V1cn/S8C8ysp7QQihKJiERhyd4fVQ18AJuCJT/+UVNfL/XQN7Ozzew3wFHgi8DV7v7jhOZLgMMV6w4X1yfZCIyVLSq2JiJR8GNT2FTtoy02NdXUk7uphz7wGHAO8CbgC8AdZvbaJj7/FqC3bDmtic8tIjJrNq+AF6xmGy8YNq95UZ166Lv7C+7+hLvvdveNwAjw4YTmh4DFFesWF9cnPf9Rdx8vLaiOvohE5NCq1+EJSewFOPTa5o5TST30qygACxIeGwYuqVi3luRzACIiUev9u2/zwisXTAt+L8ALr1xA79e/1dTXS3uc/hYzu9DMlheP7W8BLgK+Xnx8e3FdyVbgUjP7mJm9xsw+SRjqeVu7+y4i0gzdy07lhYd+xuiafl582XyOnVzgxZfNZ3RNPy889DO6l53a1NdLu57+K4HtwCmEk6yPAG9393uLjy8DXjqD4e67zOx9wM3AZ4CfAle5+6Nt7bWISBN1LzuV7uKwTD82hc0rtOzkY3Tj9FtN4/RFpNNkcpy+iIi0nkJfRCRHFPoiIjmi0BcRyRGFvohIjij0RURyRKEvIpIjCn0RkRxR6IuI5IhCX0QkRxT6IiI5otAXEckRhb6ISI4o9EVEckShLyKSIwp9EZEcUeiLiOSIQl9EJEcU+iIiOaLQF5HqcjZ/dl7MT7sDIhKRyQkY2QSjQzA1CYUu6FsH/Zuhqzvt3kkTKPRFJJicgHtWw9heYOp36x8fhMP3w9uGFfwdQId3RCQY2TQ98CHcH9sLIzel0StpMoW+iASjQ0wP/JIpGN3Zzt5Iiyj0RSSctJ2arN1malIndzuAQl9EwCyctK2l0BXaSaYp9EUk6FtHciQUoO+KdvZGWkShLyJB/2boXcX0WCiE9f03p9EraTKFvogEXd1hWObKAVi4HE7qC7crBzRcs4OY5+zEjJn1AGNjY2P09PSk3R2ReLnrGH5GjI+P09vbC9Dr7uO12mpPX0SqU+B3JIW+iEiOKPRFRHJEoS8ic5Oz84JZp9AXkcZNTMCGDbBiBSxdGm43bAjrJWqphr6ZbTSz75nZhJk9Y2Y7zOzMGbZZb2ZesRxpV59Fcm9iAlavhsFB2L8fRkfD7eBgWK/gj1rae/pvAQaBNwNrgS7gHjNbOMN248ApZcurWtlJESmzaRPs3QtTFcXZpqbC+ptUjTNmqYa+u1/q7re7+4/cfQRYDywDzpt5Uz9UthxueWdFJBgamh74JVNTsFPVOGOW9p5+pd7i7bMztFtkZj83swNmdpeZvS6poZktMLOe0gLoskKR2XKHyRmqcU6qGmfMogl9MysAtwLfcfdHazR9DPgAcCXwfsK/YZeZnZbQfiMwVrYcbFafRXLHDLpmqMbZpWqcMYsm9AnH9s8C3lurkbsPu/t2d9/j7g8C7wJ+AVyXsMkWwi+I0pL05SAi9Vi3DgoJ0VEowBWqxhmzKELfzG4DLgfe6u4N7Ym7+yTwA+CMhMePuvt4aQE0tEBkLjZvhlWrpgd/oRDW36xqnDFLe8imFQP/auBid983i+eYB5wNPN3s/olIFd3dMDwMAwOwfDn09YXbgYGwvlunzWKWapVNM/s88D7C8fnHyh4ac/fni222A6PuvrF4/xPAd4EngJOBjwNXAee5+4/reE1V2RRpJlXjTF0jVTbnt6dLiT5UvH2gYv2/B24v/vcyjp+t+feALwFLgF8Du4E19QS+iLSAAj9TVE9fRCTjVE9fpJlytmMknS3twzsicZqcgJFNMDoEU5NQ6AoTh/dv1rSBkmkKfZFKkxNwz2oY28txp5MeH4TD92u+WMk0Hd4RqTSyaXrgQ7g/thdGVFBMskuhL1JpdIjpgV8yBaMqKCbZpdAXKecejuHXMqWCYpJdCn2RcmbhpG0tBRUUk+xS6ItU6ltH8p9GAfpUUEyyS6EvUql/M/SuYvqfRyGs71dBMckuhb5Ipa7uMCxz5QAsXA4n9YXblQMarimZpzIMIjNRQTGJnMowiDSTAl86iEJfRCRHFPoiIjmi0BcRyRGFvohIjij0RURyRKEvIpIjCn0RkRxR6IuI5IhCX0QkRxT6IiI5otAXEckRhb6ISI4o9CU9OavwKhKD+Wl3QHJmcgJGNoXJx6cmw9SDfevCxCWdUqdepZglYtrTl/aZnIB7VsPjg/Db/fD8aLh9fDCsn5xIu4ezNzEBGzbAihWwdGm43bAhrBeJiCZRkfZ5eEMIeKaqPFgIM1Odv7XdvZq7iQlYvRr27oWpsn9boQCrVsHwMHR3yK8YiZImUZE4jQ5RPfAJ60d3trM3zbNp0/TAh3B/71646aZ0+iVShUJf2sM9HMOvZWoymyd3h4amB37J1BTszOiXmXQkhb60h1k4aVtLoSt7J0DdYXKGL7PJjH6ZSUdS6Ev79K0j+SNXgL4r2tmb5jCDrhm+zLoy+GUmHUuhL+3Tvxl6VzH9Y1cI6/tvTqNXc7duXThpW02hAFdk8MtMOpZG70h7TU7AyE3hpO1L4/SvCIGf1XH6Gr0jKWtk9I5CX9LTSRcxTUyEUTo7d4Zj+F1dYQ//5psV+NJymQl9M9sIvAt4DfA8sAv4L+7+2AzbXQN8GlgO/LS4zbfqfE2FvrRWJ32ZSSZkaZz+W4BB4M3AWqALuMfMFiZtYGZrgG8AXwHOBXYAO8zsrJb3VqQeCnyJWFSHd8zsFcAzwFvc/X8ltLkTWOjul5et+y6wx92vr+M1tKcvIh0lS3v6lXqLt8/WaLMauK9i3d3F9dOY2QIz6yktgA6wikhuRRP6ZlYAbgW+4+6P1mi6BDhcse5wcX01G4GxsuXg3HoqIpJd0YQ+4dj+WcB7m/y8Wwi/IErLaU1+fhGRzIiinr6Z3QZcDlzo7jPtiR8CFlesW1xcP427HwWOlr3WHHoqItI8aQz0SnVP34LbgKuBi919Xx2bDQOXVKxbW1wvIhK1tKdeSHuc/ueB9wFXAuVj88fc/flim+3AqLtvLN5fAzwI3Ah8k3A46L8Cb5jhXEDpNTV6R0RS0aqLt7M0eudDhOPsDwBPly3vKWuzDDildMfddxG+KD4IjADvBq6qJ/BFRNIUw9QLUY3Tbwft6YtIWlasgP37kx9fvhz21XOQu0KW9vQlJjnbARBpp1imXohi9I6kaHICRjaFqQxfqnq5LpRBzmrVS5EIxTL1gvb082xyAu5ZHSYr/+1+eH403D4+GNZPtmk4gUhOxDD1gkI/z0Y2wdhepk9WPhXWj2hCb5Fm2rw5jNKpDP7S6J2b2zCPkEI/z0aHmB74JVNhohMRaZru7jAsc2AgnLTt6wu3AwPtm2tHx/Tzyj0cw69lalK14UWarLsbtm4NS+6uyJUUmYWTtrUUNKG3SCul8eel0M+zvnUkfwQKYe5aEekoCv08698MvauY/jEohPX9bTirJCJtpdDPs65ueNswrByAhcvhpL5wu3IgrNc4fZGOozIM8jtZOGmbhT6KtJnKMMjsxBqmadeiFekg2tOXuLWqFq1IB9GevnSOGGrRinQQhb7EbWhoeuCXTE3BTl01LNIIhb7EK5ZatCIdRKEv8YqlFq1IB1HoS9xiqEUr0kEU+hK3GGrRinQQhX7Wdfrx7Bhq0Yp0EI3Tz6I8T3GoK3JFpmlknL7q6WdNaYrDyhmvHh+Ew/d3fs0cBb5kUEz7Kjq8kzWa4lAkE2KtHqLDO1lz14oweXmShcvhyn3t6o2IVNHu6iEqw9CpGpniUERSE3P1EIV+lmiKQ5FMiLl6iEI/azTFoUjUYq8eotDPGk1xKBK12KuHKPSzRlMcikQv5uohGr2TdTENABYRQKN3pJUU+CLRibl6iPb0RURarNU/yLWnLyISkZh+kCv0RURypGmhb2arzOxnDW5zoZkNmdlTZuZmdtUM7S8qtqtclsyp8yIiOdHMPf0TgFc1uM1CYAS4ocHtzgROKVueaXB7EZFcqru0spndMkOTVzT64u7+beDbxedvZNNn3P25Rl8vE7I0BDNLfRURoLF6+h8G9gBJZ4YXzbk39dtjZguAR4FPuvt32vjazZelSVEmJkI1qaGhcC15V1e4EmXzZs1iJUL8+0J1D9kLine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAFtCAYAAADrr7rKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAtE0lEQVR4nO3df3RU9Z3/8ec7IeFXEtBVCSA0oShFUUDpKrQC/qBSW0C3WlztD467W3/loP3ybS1fXGstEd2eb1esqfptu4u0q+X0nF0g1q6oLGmtVOsPsJYIWkEh8sMfBeIPQmQ+3z/uTBwmmUkmmZl779zX45x7hrnzuZk3w/DO537u574/5pxDRESiocTvAEREpHCU9EVEIkRJX0QkQpT0RUQiRElfRCRClPRFRCJESV9EJEKU9EVEIqSf3wEUmpkZMAJo9TsWEZEcqgTedN3ccRu5pI+X8Hf5HYSISB6cCLRkahDFpN8KsHPnTqqqqvyORUSkzw4ePMioUaOgByMYUUz6AFRVVSnpi0jk6EKuiEiEKOmLiESIkr6ISIREdkw/E+ccH330EUeOHPE7FAFKS0vp168f3mxbEekLJf0Uhw8fZvfu3XzwwQd+hyJJBg0axPDhwykvL/c7FJFQU9JPEovF2L59O6WlpYwYMYLy8nL1Ln3mnOPw4cO89dZbbN++nZNOOomSEo1KSvFyR2JYaf6+474mfTO7FrgWqInv+jNwm3PuNxmOuQz4fvyYV4CbnHOP5CKew4cPE4vFGDVqFIMGDcrFj5QcGDhwIGVlZbz++uscPnyYAQMG+B2SSE61vvEmB674PNXNf8ZiDldi7Bl/KkMe/A2Vo0fk9L387jLtAr4DnAlMAdYDa8zs1K4am9k04CHgZ8BkYDWw2swm5DIo9SSDR/8mUqxa33iT8rPGMPKpF+n37hFK98fo9+4RRm58kfKzxtD6xps5fT9f/yc55xqdc484515xzm1zzi0B3gPOTnPIDcB/O+d+4Jxrds79M/A8UFeomEVEcunAFZ+nfG8bllIxx2JQvq+NA1delNP3C0z3ycxKzexyYDCwMU2zqcDjKfseje9P93P7m1lVYsMrSiQiEgjVzX/ulPATLAbVW/6c0/fzPemb2Wlm9h7QBtwHXOKc25KmeTWwN2Xf3vj+dBYDB5I2FVsTkUBwR2JYLGNRTCwWwx2J5ew9fU/6wFZgEnAWcC/wgJmdksOfvwwYkrSdmMOfHSh79uzhhhtuYOzYsQwYMIBhw4bxmc98hnvvvbdjCmpNTQ1mhpkxcOBAampq+PKXv8z69et9jl4keqy0BFeSeYagK7GczubxPek75w475151zj3nnFsMbMYbu+/KHmBYyr5h8f3pfn6bc+5gYqPAdfQzV7bOnddee43Jkyezbt06br/9dl544QU2btzIt7/9bR5++GEef/zjUbHbbruN3bt3s3XrVlauXMnQoUO54IILqK+vL0ywItJhz/hTcWkysSuBPafkdJ5KIOfplwD907y2ETgfuCtp3yzSXwPwRWsrLFkCjY3Q3g5lZTBnDtTXQ2Werihcd9119OvXj2effZbBgwd37B8zZgzz5s0jeV2FyspKqqu9EbHRo0czffp0hg8fzi233MKll17KuHHj8hOkiHQy5MHfcPisMZTva8OSRnFcCRw+oT9D/iMnM9I7+NrTN7NlZjbdzGriY/vLgJnAf8RfXxnfl7AcmG1mi8zsU2Z2K95Uz3sKHXs6ra0wdSo0NMCOHdDS4j02NHj7W/NwnvHOO++wbt06rr/++qMSfrLubjK74YYbcM6xZs2a3AcoImlVjh7B4adfo2XaRD46th9Hhpbw0bH9aJk2kcNPv1Z08/RPAFbijes/AXwauNA591j89dHA8ERj59xTwBXAN/CGgS4FLnbOvVTIoDNZsgSamyGWct0lFvP233xz7t/z1VdfxTnXqYd+3HHHUVFRQUVFBTfddFPGn3HsscdywgknsGPHjtwHKCIZVY4ewYm/20S/d9opebudfu+0c+LvNuU84YPPwzvOuX/o5vWZXez7FfCrfMXUV42NnRN+QiwGa9fC8uWFieWZZ54hFotx5ZVX0tbW1m1755zKToj4LJ8lGCCYY/qh5Zw3hp9Je7vXLpe5dezYsZgZW7duPWr/mDFjAK+MQXfeeecd3nrrLWpra3MXmIgEjt/DO0XFzLtom0lZWW4TPsDf/M3fMGvWLO655x7ef//9Xv2M5cuXU1JSwsUXX5zb4EQkUJT0c2zOHEhXJqakBObOzc/7/vjHP+ajjz5iypQprFq1iubmZrZu3covfvELXn75ZUpLSzvatra2smfPHnbu3Mlvf/tbvvGNb7B06VLq6+sZO3ZsfgIUkUAwV6iJ5AERL8Vw4MCBA50WRj906BDbt2+ntra215UcE7N3Ui/mlpTA+PGwcWP+pm3u3r2b22+/nV//+tfs2rWL/v37c8opp3DZZZdx3XXXMWjQIGpqanj99dcBKC8vp7q6mrPPPptrrrmGc889Nz+B5UAu/m1EitXBgwcZMmQIwJD4/UhpKeknyVViaW31ZumsXfvxPP25c2Hp0vwl/GKnpC+SXjZJXxdy86Cy0puhs3x57i/aioj0hcb080wJX0SCRElfRCRClPRFRCJESV9EJEKU9EVEIkRJX0QkQpT0RUQiRElfRCRClPSLxIIFCzAz7rjjjqP2r169Om/lkm+99VYmTZqUl58tIvmhpF9EBgwYwJ133slf//pXv0MRkYBS0s+3AtY2uuCCC6iurmbZsmVp2zz55JOcc845DBw4kFGjRrFw4cKOcsz33HMPEyZ8vAhz4izhvvvuO+o9br75ZlasWMH3vvc9Nm/ejJlhZqxYsQKAN954g3nz5lFRUUFVVRVf/vKX2bt3b8fPSJwh/PznP6empoYhQ4Zw+eWX05qPtSRF5ChK+vnQ3grPLoQ1tbB6lPf47EJvfx6VlpZy++2386Mf/Yhdu3Z1ev0vf/kLs2fP5ktf+hIvvvgiq1at4sknn6Surg6AGTNmsGXLFt566y0AmpqaOO6449iwYYP312pvZ+PGjcycOZP58+ezaNEiTj31VHbv3s3u3buZP38+sViMefPm8e6779LU1MRjjz3Ga6+9xvz58zvFsnr1ah5++GEefvhhmpqaOg1NiUjuKennWnsrrJsK2xrg/R3wYYv3uK3B25/nxH/JJZcwadIkvvvd73Z6bdmyZVx55ZXceOONnHTSSUybNo27776blStXcujQISZMmMCxxx5LU1MTABs2bGDRokUdz5955hna29uZNm0aAwcOpKKign79+lFdXU11dTUDBw7kiSee4E9/+hMPPvggZ555JmeddRYrV66kqamJP/7xjx2xxGIxVqxYwYQJEzjnnHP46le/yhNPPJHXz0ZElPRzb/MSONAMpC6UG/P2b87Dyugp7rzzTh544AGam5uPDm3zZlasWNGxWHpFRQUXXnghsViM7du3Y2ZMnz6dDRs2sH//frZs2cJ1111HW1sbL7/8Mk1NTXz6059m0KBBad+7ubmZUaNGMWrUqI59p5xyCkOHDj0qnpqaGiqT6kwPHz6cffv25fBTkD6LWNn1qFDSz7WWRjon/IQYtKzNewjTp0/nwgsvZPHixUftf++997j66qvZtGlTx7Z582ZeeeUVPvnJTwIwc+ZMNmzYwO9+9zsmT55MVVVVxy+CpqYmZsyYkZMYy1LWlTQzYulWlJfC8WloUgpH9fRzyTmIdbMyeiwPK6N34Y477mDSpEmMGzeuY98ZZ5zBli1bMi6JOGPGDG688UZ+9atfMXPmTMD7RfD444/z+9//nkWLFnW0LS8v58iRI0cdP378eHbu3MnOnTs7evtbtmxh//79nHLKKTn8G0rOJYYmU89UtzXA3vXwuY1QplWAwk49/Vwyg5JuVkYvycPK6F047bTTuPLKK7n77rs79t1000089dRT1NXVsWnTJl555RXWrFnTcSEX4PTTT+eYY47hwQcfPCrpr169mra2Nj7zmc90tK2pqWH79u1s2rSJt99+m7a2Ni644IKO937++ed55pln+NrXvsaMGTOYMmVK3v/e0gcBGJqU/FPSz7WRc0j/sZbAyDytjN6F22677aghk9NPP52mpia2bdvGOeecw+TJk7nlllsYMWJERxsz45xzzsHM+OxnP9txXFVVFVOmTGHw4MEdbb/0pS8xe/Zszj33XI4//ngeeughzIw1a9ZwzDHHMH36dC644ALGjBnDqlWrCvb3ll4KwNCk5J/WyE2Sk3VY050iUwJDxusUuZe0Rm6eOeeN4X/Ykr7NwJFw8U4tBxdA2ayRq55+rpVVeon95DoYXOP9Rxlc4z1XwpegCtDQpOSXLuTmQ1klTFnubVoZXcJi5Bzvom2XQzyFHZqU/FFPP9+U8CUsJtZ7Q5Cd0kJ8aHLiUj+ikhxT0hcRj4YmI0HDOyLyMQ1NFj319EWka0r4RUlJX0QkQpT0RUQiRElfRPomYjd4hp2Svohkr7UVFi6E2loYNcp7XLjQ2y+B5mvSN7PFZvZHM2s1s31mttrMxnVzzAIzcynboULFHFSJhdHNjPLycsaOHcttt93GRx99BIBzjp/85CdMnTqVqqoqKioqOPXUU7nhhht49dVXfY5eQqW1FaZOhYYG2LEDWlq8x4YGb78Sf6D53dOfATQAZwOzgDJgnZkNzngUHASGJ22fyGeQfVLAU9/Zs2eze/duXnnlFRYtWsStt97KD37wA5xzXHHFFSxcuJCLLrqIdevWsWXLFn72s58xYMAAli7VTTeShSVLoLkZUtc/iMW8/TerGmeQ+TpP3zk3O/m5mS0A9gFnAr/NfKjbk8fQ+qa11fuP0dgI7e1QVgZz5kB9PVTm7waX/v37U11dDcC1117Lf/3Xf7F27Vpqa2v55S9/yZo1a5g79+Nb6UePHs3ZZ59N1IruSR81NnZO+AmxGKxdC8uXFzYm6bGLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 400x400 with 1 Axes>"
|
||||
]
|
||||
@@ -556,19 +552,18 @@
|
||||
"updatesPG = onp.asarray(updatesPG) \n",
|
||||
"\n",
|
||||
"axes = plt.figure(figsize=(4, 4), dpi=100).gca()\n",
|
||||
"axes.scatter(historyGD[:,0], historyGD[:,1], lw=0.5, color='blue')\n",
|
||||
"axes.scatter(historyNt[:,0], historyNt[:,1], lw=0.5, color='orange')\n",
|
||||
"axes.scatter(historyPG[:,0], historyPG[:,1], lw=0.5, color='red')\n",
|
||||
"axes.scatter(historyGD[:,0], historyGD[:,1], lw=0.5, color='blue', label='GD')\n",
|
||||
"axes.scatter(historyNt[:,0], historyNt[:,1], lw=0.5, color='orange', label='Newton')\n",
|
||||
"axes.scatter(historyPG[:,0], historyPG[:,1], lw=0.5, color='red', label='PG')\n",
|
||||
"axes.scatter([0], [0], lw=0.25, color='black', marker='x') # target at 0,0\n",
|
||||
"axes.set_xlabel('x0')\n",
|
||||
"axes.set_ylabel('x1')"
|
||||
"axes.set_xlabel('x0'); axes.set_ylabel('x1'); axes.legend()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This illustrates that the PG variant does significantly better than Newton's method. It yields a trajectory that is better aligned with the _diagonal_, and its final state is closer to the origin. A key ingredient here is the inverse function for $\\mathbf{y}$, which yielded higher order terms than the second-order approximation for Newton's method. Despite the simplicity of the problem, Newton's method has problems finding the right search direction. For PG, on the other hand, the higher order information yields an improved direction for the optimization.\n",
|
||||
"This illustrates that the inverse simulator variant, PG in red, does even better than Newton's method in orange. It yields a trajectory that is better aligned with the ideal _diagonal_ trajectory, and its final state is closer to the origin. A key ingredient here is the inverse function for $\\mathbf{y}$, which provided higher order terms than the second-order approximation for Newton's method. Despite the simplicity of the problem, Newton's method has problems finding the right search direction. For the inverse simulator update, on the other hand, the higher order information yields an improved direction for the optimization.\n",
|
||||
"\n",
|
||||
"This difference also shows in first update step for each method: below we measure how well it is aligned with the diagonal."
|
||||
]
|
||||
@@ -619,7 +614,7 @@
|
||||
"\n",
|
||||
"A first thing to note is that for PG, we explicitly map from $L$ to $\\mathbf{y}$, and then continue with a mapping to $\\mathbf{x}$. Thus we already obtained the trajectory in $\\mathbf{y}$ space, and not coincidentally, we already stored it in the `historyPGy` list above.\n",
|
||||
"\n",
|
||||
"Let's directly take a look what PG did in $\\mathbf{y}$ space:"
|
||||
"Let's directly take a look what the inverse simulator did in $\\mathbf{y}$ space:"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -630,7 +625,7 @@
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Text(0, 0.5, 'z1')"
|
||||
"<matplotlib.legend.Legend at 0x7fe604e57fd0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 11,
|
||||
@@ -639,7 +634,7 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAGDCAYAAAAyH0uIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAc+klEQVR4nO3de5RdZZnn8e9DAhWXpArtQuju6KCFTcbbIATBTA9gFNBup9XltVHHmL4oLa4WZ8YWWwGdQYYZF6HpMC0daZHuRFg9beOIIzReclOCNN4K76QlQnMtIaeIQBHDM3/sU+RUUVWpqpxT+7yp72etvU6dd7/77OfsVH7ZefctMhNJUnkOqLsASdLsGOCSVCgDXJIKZYBLUqEMcEkqlAEuSYUywCWpUAa4JBXKAJekQhngklQoA1ySCmWAS1KhDHBJKpQBrmJExMsiIiPidRPMO70576VTLH9gRJwbET+NiEcj4hcRsSUiTmnpc0VE7IyI50TE9RHxy4i4KyLOiYgY93n/JSK+0fycRyLiloh4wyTrfltEfDMiHo6IByNiU0ScOq7PqyJic3OdD0XEFyPi+TPfUpovDHCVZANwB/DWCea9FdiWmTdOsfx5wLnA14AzgfOBnwPHjOu3ALgOuBf4AHAL8NHm1OpPgW8D5wAfAn4F/H1E/G5rp4g4F/hbYFez77nN77Gipc/bgS8CO4E/A/4b8DxgS0QcMcV30nyWmU5OxUzAx4FHgb6WtkOpwvG8vSz7HeDavfS5Akjgkpa2AK4FRoD+lvanjFv2QGAQ+EpL25HAbuBzwAHj+kfz9WDgQeCvx80/DNgxvt3JaXRyD1yluRLoAVqHKt4MLAT+bi/L7gCeHxHPncZ61oz+kJnZfH8Q8IqW9kdGf46IpwF9wGbG7tG/lup/uh/LzMdbV9D8XIBTgEOAz0ZE/+hEFfw3AS+bRr2ahxbWXYA0E5n5o4i4mWrI5PJm81uBrZl5214WPwf4PPCTiLiVapjkbzPze+P6PQ78y7i2nzRfjxhtiIhXAx8Gjqb6R+WJMlt+Hmh+3g+mqGv0H5SvTjJ/eIplNY8Z4CrRlcBfRMQSquA8gWpMe0qZuSkiBoDXAKcCfwicFRHvzsxPzaSAiPgPwP8FNgF/AtxNNYzzTuD0mXwWe45FvR24Z4L5v5rh52meMMBVoquAi4DfB55CFZxXT2fBzHwA+DTw6Yg4mCqAzwNaA/wA4Dns2esG+K3m6+3N19dTjcWflpkjo50i4p3jVrmt+XnPoxqDn8i25ut9mfnl6XwPCTwLRQXKzCHgS8DbqIZPrmu2TSkifm3c5+wEbmPs8MeoJ/bom6cPnkn1D8VXms27qYZKFrT0O4JqzLvVNVRDKOdExJi/by2nJV5PNUzyoYg4cIK6D53yi2necg9cpboS+D/Nnz8yzWV+EBEbqE4LfABYRnUwdM24fo8Cr4yIz1AdRHwV8LvAxzPz/mafLwLvB66LiPXAM4D3UP2D8KLRD8rM2yLi/GaNmyPic1RnsxwH3AWcnZnDEXEG1amG34qIq4D7gWc11/t1pjFEpPkn9hwIl8oREQdRjRcfAByemY9OY5k/B36PajikB9hOFZr/KzN3NftcQRXqLwL+Cvht4CHgk4w7kyQiVgEfpAranwEXUh3kPDczx1/0807gvVRDKQ8D3wP+e+uQSUSc3Py8E5r1/SvVWS1rMvOWaW4azSMGuIoUEQup9mC/kJl/0MbPvQJ4Q2Ye3K7PlDrFMXCV6rVUF/BcWXMdUm0cA1dRIuJ4quGNjwDfzsyNNZck1cY9cJXmDKqx6fuA/1RzLVKtHAOXpEK5By5JhTLAJalQRR/EbF7J9htU5+lK0v5iMXBX7mWMu+gApwrvO+suQpI6YAnVxVyTKj3AHwK444476O3trbsWSdpnw8PDPPOZz4RpjCyUHuAA9Pb2GuCS5h0PYkpSoQxwSSqUAS5JhTLAJalQBrgkFcoAl6RO6fC9pgxwSeqE3bth2bLqtUMMcEnqhM2bYXAQtmzp2Cr2iwt5JKkrDA1Bo1H9vHo17NpVvS5ZUrX19UF/f9tW5x64JLXL+efD0qWwfDls2lS1bdxYvV+6tJrfRga4JLXL6tVw2WWwYAHs2FG17dgBCxfC2rXV/DYywCWpnVatghUrxratWAErV7Z9VY6BS1I7ZVbDJv39MDAA27bBhg1Ve0RbV+UeuCS10+Ag7NwJ69fD1q2wbl31fnCw7asq+qHGEdELNBqNhreTldQdRkaqqTWTGg1YtAh6eva6+PDwMH19fQB9mTk8VV+HUCSpnXp6nhzUVSC3nUMoklQoA1ySCmWAS1KhDHBJKlTtAR4RvxkRfxcRv4iIRyJiMCKW1V2XJHW7Ws9CiYinAV8Hvga8CrgfeC7wYJ11SVIJ6j6N8M+AOzLznS1tP6urGEkqSd1DKL8H/HNE/H1E3BcR346IP5qsc0T0RETv6AQsnrtSJam71B3gzwHOAH4KnAb8FXBJRLxjkv5nA42W6c65KFKSulGtl9JHxGPAP2fm8pa2S4DjMvOlE/TvAVovcVoM3Oml9JL2FyVdSn838INxbT8EXj9R58wcAUZG30eb7+wlSSWpewjl68BR49p+C9heQy2SVJS6A3w1cEJEfCgijoyI04E/Bi6tuS5J6nq1Bnhm3gy8Dvh94FbgI8D7MnNdnXVJUgnqHgMnM68Frq27DkkqTd1DKJKkWTLAJalQBrgkFcoAl6RCGeCSVCgDXJIKZYBLUqEMcEkqlAEuSYUywCWpUAa4JBXKAJekQhngklQoA1ySCmWAS1KhDHBJKpQBLkmFMsAlqVAGuCQVygCXpEIZ4JJUKANckgplgEtSoQxwSSqUAS5JhTLAJalQBrgkFcoAl6RCGeCSVCgDXJIKZYBLUqEMcEkqVK0BHhHnRUSOm35UZ02SVIqFdRcAfB94Rcv7X9VViCSVpBsC/FeZeU/dRUhSabphDPy5EXFXRPxLRKyLiGdN1jEieiKid3QCFs9hnZLUVeoO8JuAlcArgTOAZwObI2KyYD4baLRMd85BjZLUlSIz667hCRFxCLAdeH9mXj7B/B6gp6VpMXBno9Ggt7d3boqUpA4aHh6mr68PoC8zh6fq2w1j4E/IzB0R8RPgyEnmjwAjo+8jYq5Kk6SuU/cQyhgRcTAwANxddy2S1O3qPg/8ExFxUkQcERHLgX8EdgOfrbMuSSpB3UMoS6jC+teA+4EtwAmZeX+tVUlSAWoN8Mx8S53rl6SSddUYuCRp+gxwSSqUAS5JhTLAJalQBrgkFcoAl6RCGeCSVCgDXJIKZYBLUqEMcEkqlAEuSYUywCWpUAa4JBXKAJekQhngkvboomfkau8McEmV3bth2bLqVUUwwCVVNm+GwUHYsqXuSjRNdT9STVKdhoag0ah+Xr0adu2qXpcsqdr6+qC/v776NCX3wKX57PzzYelSWL4cNm2q2jZurN4vXVrNV9cywKX5bPVquOwyWLAAduyo2nbsgIULYe3aar66lgEuzXerVsGKFWPbVqyAlStrKUfT5xi4NN9lVsMm/f0wMADbtsGGDVV7RN3VaQrugUvz3eAg7NwJ69fD1q2wbl31fnCw7sq0F5EFn7gfEb1Ao9Fo0NvbW3c5UplGRqqp9e9QowGLFkFPT311zVPDw8P09fUB9GXm8FR9HUKR5ruenicHdRUg6nIOoUhSoQxwSSqUAS5JhTLAJalQBrgkFcoAl6RCGeCSVCgDXJIK1TUBHhEfjIiMiIvrrkWSStAVAR4RxwHvAr5Xdy2SVIraAzwiDgbWAX8EPFhzOZJUjNoDHLgU+GJmfnlvHSOiJyJ6RydgcefLk6TuVOvNrCLiLcAxwHHTXORs4NzOVSRJ5ahtDzwingn8BfDWzHx0motdAPS1TEs6VJ4kdb0698CPBZ4BfCv2PPVjAXBiRJwJ9GTm7tYFMnMEGBl9Hz4tRNI8VmeAfwV44bi2TwM/Ai4cH96SpLFqC/DMfAi4tbUtIn4J/CIzb514KUnSqG44C0WSNAtd9Ui1zDy57hokqRTugUtSoQxwSSqUAS5JhTLAJalQBrgkFcoAl6RCGeCSVCgDXJIKZYBLUqEMcEkqlAEuSYUywCWpUAa4JBXKAJekQhngklQoA1ySCmWAS1KhDHBJKpQBLkmFMsAlqVAGuCQVygCXpEIZ4JJUKANckgplgEtSodoa4BHx7yJidzs/U5I0sU7sgUcHPlOSNM7CmXSOiM/tpUsfkLMvR+oymRDuk6g7zXQP/D8Ci4DGJNPOtlYn1Wn3bli2rHqVutCM9sCBHwL/kJmXTzQzIo4GXr2vRUldYfNmGByELVvgpJPqrkZ6kpkG+C3AMcCEAQ6MAD/fp4qkOg0NQaNR/bx6NezaVb0uWVK19fVBf3999UktInP6Q9YR0QMsyMyHO1fS9EVEL9BoNBr09vbWXY72B2edBWvWwNOfDo89Bjt2wCGHwEEHwQMPwJlnVoEudcjw8DB9fX0AfZk5PFXfGY2BZ+ZIZj4cEV+NiHPHz4+Ip0XEV6f7eRFxRkR8LyKGm9ONEfGqmdQktdXq1XDZZbBgQRXeUL0uXAhr1xre6iqzPY3wZODMiLgmIp7a0n4QMJPBwjuBDwLHAsuArwKfj4jnz7Iuad+tWgUrVoxtW7ECVq6spRxpMvtyHvgrgMOBrRFxxGw+IDO/kJn/LzN/mpk/ycw/pzqT5YR9qEvaN5mwcWM11n388dXrhg1Vu9RF9iXA76ba2x4Ebo6Ik/elkIhYEBFvAZ4K3DhJn56I6B2dgMX7sk5pQoODsHMnrF8PW7fCunXV+8HBuiuTxpjpWSijEqoxceD0iPgwcB1w4Uw/KCJeSBXYi6j2vl+XmT+YpPvZwJPG3qW2Ouoo2L4dRg+Mn3oq3H47LFpUa1nSeDM6C+WJhSIeBw7PzPta2l4PfAZ4SmYumMFnHQQ8i+oqzjcAfwicNFGIN8+C6WlpWgzc6VkLine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAGDCAYAAAAyH0uIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAgR0lEQVR4nO3df5xddX3n8deHSTIBkploJ4A0sOJAkxV1rQb5sV2BofzQulYfVsHYrjH9oVSsUndboxZSVsq69kEojWtppCJtArRbFUUrP6r5pURdxDqUCpKGFEogDJA7xMAkhO/+ce6QO8PMZGYy9577nft6Ph7nced+7/fc+7knk3dOvud7zomUEpKk/BxSdgGSpMkxwCUpUwa4JGXKAJekTBngkpQpA1ySMmWAS1KmDHBJypQBLkmZMsAlKVMGuCRlygCXpEwZ4JKUKQNc2YiIMyMiRcTbR3htSfW1U8dYf2ZEXBoRP42IZyPiiYjYFBFn1/S5LiJ2RcQrIuLWiPhZRDwSEZdERAx7v/8eEd+tvs8zEXFXRPzaKJ/96xHx/YjYHRFPRcSGiDhnWJ83RcTG6mc+HRFfj4gTJ76l1CoMcOVkHfAQ8J4RXnsPsCWldOcY668ALgW+DVwEXA78G/C6Yf3agG8CjwF/ANwF/HF1qfVh4G7gEuDjwHPA30XEr9R2iohLgb8G9lb7Xlr9Hj01fX4D+DqwC/hD4H8CrwQ2RcTLx/hOamUpJReXbBbgT4Bngc6atvkU4bjiAOv+CLjlAH2uAxJwdU1bALcAA0BXTfuhw9adCfQC/1jTdjywD/gScMiw/lF9nAM8BfzlsNePBHYOb3dxGVzcA1durgfagdqhivOBGcDfHGDdncCJEXHCOD5n1eAPKaVUfT4L+OWa9mcGf46IlwCdwEaG7tG/jeJ/upellJ6v/YDq+wKcDcwDboiIrsGFIvi/B5w5jnrVgmaUXYA0ESmln0TEDyiGTK6tNr8H2JxSeuAAq18C3AzcHxH3UAyT/HVK6cfD+j0P/Ouwtvurjy8fbIiItwCfBF5L8Y/KC2XW/Nxdfb97x6hr8B+Ub43yev8Y66qFGeDK0fXAn0XEAorgPIViTHtMKaUNEdEN/CpwDvBbwMUR8YGU0ucnUkBE/Bfgq8AG4HeB7RTDOO8Dlkzkvdh/LOo3gEdHeP25Cb6fWoQBrhzdCFwJvBs4lCI4bxrPiimlJ4EvAF+IiDkUAbwCqA3wQ4BXsH+vG+AXqo8PVh/fQTEWf25KaWCwU0S8b9hHbqm+3yspxuBHsqX6uCOldMd4vocEzkJRhlJKfcA/AL9OMXzyzWrbmCLi54a9zy7gAYYOfwx6YY++On3wIop/KP6x2ryPYqikrabfyynGvGt9hWII5ZKIGPL3rWZa4q0UwyQfj4iZI9Q9f8wvppblHrhydT3wf6s//9E417k3ItZRTAt8ElhMcTB01bB+zwLnRcQXKQ4ivgn4FeBPUkqPV/t8Hfh94JsRsRY4AvggxT8Irxl8o5TSAxFxebXGjRHxJYrZLCcBjwDLU0r9EXEhxVTDH0bEjcDjwLHVz/0O4xgiUuuJ/QfCpXxExCyK8eJDgKNSSs+OY51PAG+lGA5pB7ZRhOZnUkp7q32uowj11wCfA34JeBr4C4bNJImIZcDHKIJ2K/BpioOcl6aUhp/08z7gQxRDKbuBHwOfqh0yiYgzqu93SrW+f6eY1bIqpXTXODeNWogBrixFxAyKPdivpZR+cwrf9zrg11JKc6bqPaV6cQxcuXobxQk815dch1Qax8CVlYg4mWJ444+Au1NK60suSSqNe+DKzYUUY9M7gP9Wci1SqRwDl6RMuQcuSZkywCUpU1kfxKyeyXY0xTxdSZou5gKPpAOMcWcd4BTh/XDZRUhSHSygOJlrVLkH+NMADz30EB0dHWXXIkkHrb+/n2OOOQbGMbKQe4AD0NHRYYBLajkexJSkTBngkpQpA1ySMjUtxsDHklLiueeeY9++fWWXUqq2tjZmzJjB/nsISMrdtA7wPXv2sH37dnbv3l12KU3hsMMO42UvexmzZs0quxRJU2DaBvjzzz/P1q1baWtr4+ijj2bWrFktu/eZUmLPnj08/vjjbN26lRNOOIFDDnH0TKq7lKCOuTNtA3zPnj08//zzHHPMMRx22GFll1O6Qw89lJkzZ7Jt2zb27NnD7Nmzyy5Jmt727YM3vAG+/31oaztw/0mY9rth7mnu57aQGmjjRujthU2b6vYR03YPXJIarq8PKpXi55UrYe/e4nHBgqKtsxO6uqbs49wlk6SpcvnlsGgRnHYabNhQtK1fXzxftKh4fQoZ4JI0VVauhGuuKca8d+4s2nbuhBkzYPXq4vUpZIA3oaVLlxIRRASzZs3i+OOP57LLLuO5554Dilklq1ev5tRTT6Wjo4M5c+Zw4okn8uEPf5gHHnig5OqlFrdsGfT0DG3r6YGlS6f8owzwiWjg7efOO+88tm/fzk9/+lM++tGPsmLFCj7zmc+QUmLJkiX83u/9Hm9+85u57bbbuPfee7n22muZPXs2n/rUpxpWo6QRpFQMm3R1wcknF4/r1tUlPzyIOV4NmBJUq729naOOOgqACy+8kC9/+ct89atf5bjjjuPGG2/k5ptv5q1vfesL/Y899lhOOeUUvMepVLLeXti1C/72b+Hss+G22+D884v217xmSj/KAB+v2ilBp5/e8I8/9NBDeeKJJ7jhhhtYuHDhkPCu1aonK0lNY+FC2LYNBi9xfc458OCDUIdzLxxCGUtfH2zZUiy1U4IG2/r66l5CSok77riDW2+9lZ6eHu6//34WLlw4pM9HPvIR5syZw5w5c1gwOF1JUjna2/eH96DOzqJ9ihngY2nwlKBat9xyC3PmzGH27Nm86U1v4vzzz2fFihUj9v3EJz7Bj370Iy655BJ27dpVt5okNReHUMayciW8+tXwyU8OnRJ09NHFlKA6HFUedOaZZ/K5z32OWbNmcfTRRzNjRvFHdcIJJ3DfffcN6Tt//nzmz5/PEUccUbd6JDUf98APpIFTgmodfvjhHH/88Rx77LEvhDfAu9/9bu677z5uvvnmun6+pObnHviB1E4J6u4uxr4HpwSVcMDwggsu4Etf+hIXXHABy5cv59xzz+XII49k27Zt3HTTTbQ1YIaMpOZQ+h54RPx8RPxNRDwREc9ERG9ELC67rhcMTglauxY2b4Y1a4rnvb2llBMR3HTTTVx11VV84xvf4KyzzmLhwoUsW7aMY445hk11vHCOpOYSZc4bjoiXAHcD3wY+BzwOnABsSSltGcf6HUClUqm86K70zz77LFu3buW44447uEunDgwUS+37VyrFlKA6HFWupynbJpLqpr+/n87OToDOlFL/WH3LHkL5Q+ChlNL7atq2llXMiNrbXxzUxcaVpFKVPYTyVuD/RcTfRcSOiLg7In57tM4R0R4RHYMLMLdxpUpScyk7wF8BXAj8FDiXYhjl6oh47yj9lwOVmuXhRhQpSc2o7AA/BPhhSunjKaW7U0p/CawGPjBK/yuAzprF0w4ltayyx8C3A/cOa/sX4B0jdU4pDQADg8/Hc90PL+60n9tCml7K3gP/DrBwWNsvANsO9o1nzpwJwO7duw/2raaNwW0xuG0k5a3sPfCVwHcj4uPA3wJvAH6nuhyUtrY25s2bx44dOwA47LDDWvZKfSkldu/ezY4dO5g3b54n+0jTRKkBnlL6QUS8nWJs+xKKKYQfSSmtmYr3H7ye9mCIt7p58+a9sE0k5a/UE3kO1lgn8tTat28fe/fubVxhTWjmzJnueUsZyOlEnoZoa2szvCRNO2UfxJQkTZIBLkmZMsAlKVMGuCRlygCXpEwZ4JKUKQNckjJlgEtSpgxwScqUAS5JmTLAJSlTBrgkZcoAl6RMGeCSlCkDXJIyZYBLUqYMcEnKlAEuSZkywCUpUwa4JGXKAJekTBngkpQpA1ySMmWAS1KmDHBJypQBLkmZMsAlKVMGuCRlygCXpEwZ4JKUKQNckjJVaoBHxIqISMOWn5RZkyTlYkbZBQD/DPxyzfPnyipEknLSDAH+XErp0bKLkKTcNMMY+AkR8UhE/GtErImIY0frGBHtEdExuABzG1inJDWVsgP8e8BS4DzgQuA4YGNEjBbMy4FKzfJwA2qUpKYUKaWya3hBRMwDtgG/n1K6doTX24H2mqa5wMOVSoWOjo7GFClJddTf309nZydAZ0qpf6y+zTAG/oKU0s6IuB84fpTXB4CBwecR0ajSJKnplD2EMkREzAG6ge1l1yJJza7seeB/GhGnR8TLI+I04MvAPuCGMuuSpByUPYSygCKsfw54HNgEnJJSerzUqiQpA6UGeErpgjI/X5Jy1lRj4JKk8TPAJSlTBrgkZcoAl6RMGeCSlCkDXJIyZYBLUqYMcEnKlAEuSZkywCUpUwa4JGXKAJekTBngkpQpA1ySMmWAS9qvie6RqwMzwCUV9u2DxYuLR2XBAJdU2LgRenth06ayK9E4lX1LNUll6uuDSqX4eeVK2Lu3eFywoGjr7ISurvLq05jcA5da2eWXw6JFcNppsGFD0bZ+ffF80aLidTUtA1xqZStXwjXXQFsb7NxZtO3cCTNmwOrVxetqWga41OqWLYOenqFtPT2wdGkp5Wj8HAOXWl1KxbBJVxd0d8OWLbBuXdEeUXZ1GoN74FKr6+2FXbtg7VrYvBnWrCme9/aWXZkOIFLGE/cjogOoVCoVOjo6yi5HytPAQLHU/h2qVGD2bGhvL6+uFtXf309nZydAZ0qpf6y+DqFIra69/cVBXQSImpxDKJKUKQNckjJlgEtSpgxwScqUAS5JmTLAJSlTBrgkZcoAl6RMNU2AR8THIiJFxFVl1yJJOWiKAI+Ik4D3Az8uuxZJykXpAR4Rc4A1wG8DT5VcjiRlo/QABz4LfD2ldMeBOkZEe0R0DC7A3PqXJ0nNqdSLWUXEBcDrgJPGucpy4NLLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 400x400 with 1 Axes>"
|
||||
]
|
||||
@@ -655,17 +650,16 @@
|
||||
"\n",
|
||||
"axes = plt.figure(figsize=(4, 4), dpi=100).gca()\n",
|
||||
"axes.set_title('y space')\n",
|
||||
"axes.scatter(historyPGy[:,0], historyPGy[:,1], lw=0.5, color='red', marker='*')\n",
|
||||
"axes.scatter(historyPGy[:,0], historyPGy[:,1], lw=0.5, color='red', marker='*', label='PG')\n",
|
||||
"axes.scatter([0], [0], lw=0.25, color='black', marker='*') \n",
|
||||
"axes.set_xlabel('z0')\n",
|
||||
"axes.set_ylabel('z1')"
|
||||
"axes.set_xlabel('z0'); axes.set_ylabel('z1'); axes.legend()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For PG we're making explicit steps in $\\mathbf{y}$ space, which progress in a straight diagonal line to the origin (which is likewise the solution in $\\mathbf{y}$ space).\n",
|
||||
"With this variant, we're making explicit steps in $\\mathbf{y}$ space, which progress in a straight diagonal line to the origin (which is likewise the solution in $\\mathbf{y}$ space).\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -675,7 +669,7 @@
|
||||
"source": [
|
||||
"Interestingly, neither GD nor Newton's method give us information about progress in intermediate spaces (like the $\\mathbf{y}$ space). \n",
|
||||
"\n",
|
||||
"For **GD** we're concatenating the Jacobians, so we're moving in directions that locally should decrease the loss. However, the $\\mathbf{y}$ position is influenced by $\\mathbf{x}$, and hence we don't know where we end up in $\\mathbf{y}$ space until we have the definite point there. (For NNs in general we won't know at which latent-space points we end up after a GD update until we've actually computed all updated weights.)\n",
|
||||
"For GD we're concatenating the Jacobians, so we're moving in directions that locally should decrease the loss. However, the $\\mathbf{y}$ position is influenced by $\\mathbf{x}$, and hence we don't know where we end up in $\\mathbf{y}$ space until we have the definite point there. (For NNs in general we won't know at which latent-space points we end up after a GD update until we've actually computed all updated weights.)\n",
|
||||
"\n",
|
||||
"More specifically, we have an update $-\\eta \\frac{\\partial L}{\\partial \\mathbf{x}}$ for GD, which means we arrive at $\\mathbf{y}(\\mathbf{x} -\\eta \\frac{\\partial L}{\\partial \\mathbf{x}})$ in $\\mathbf{y}$ space. A Taylor expansion with \n",
|
||||
"$h = \\eta \\frac{\\partial L}{\\partial \\mathbf{x}}$ yields \n",
|
||||
@@ -687,7 +681,7 @@
|
||||
"= \\mathbf{y}(x) - \\eta \\frac{\\partial L}{\\partial \\mathbf{y}} (\\frac{\\partial \\mathbf{y}}{\\partial x})^2 + \\mathcal{O}( h^2 )\n",
|
||||
"$.\n",
|
||||
"\n",
|
||||
"And $\\frac{\\partial L}{\\partial \\mathbf{y}} (\\frac{\\partial \\mathbf{y}}{\\partial \\mathbf{x}})^2$ clearly differs from the step $\\frac{\\partial L}{\\partial \\mathbf{y}}$ we would compute during the backpropagation pass in GD for $\\mathbf{y}$.\n",
|
||||
"And $\\frac{\\partial L}{\\partial \\mathbf{y}} (\\frac{\\partial \\mathbf{y}}{\\partial \\mathbf{x}})^2$ clearly differs from $\\frac{\\partial L}{\\partial \\mathbf{y}}$, which we would apply with GD when optimizing for $\\mathbf{y}$ directly.\n",
|
||||
"\n",
|
||||
"**Newton's method** does not fare much better: we compute first-order derivatives like for GD, and the second-order derivatives for the Hessian for the full process. But since both are approximations, the actual intermediate states resulting from an update step are unknown until the full chain is evaluated. In the _Consistency in function compositions_ paragraph for Newton's method in {doc}`physgrad` the squared $\\frac{\\partial \\mathbf{y}}{\\partial \\mathbf{x}}$ term for the Hessian already indicated this dependency.\n",
|
||||
"\n",
|
||||
@@ -725,7 +719,7 @@
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Text(0, 0.5, 'z1')"
|
||||
"<matplotlib.legend.Legend at 0x7fe604eb82e0>"
|
||||
]
|
||||
},
|
||||
"execution_count": 13,
|
||||
@@ -734,7 +728,7 @@
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAGDCAYAAAA26pu1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAlxElEQVR4nO3dfZycZX3v8c8vm2SDJruKiwk2UnCV5FSlah7APR6CkZDaegr4hEV7GmNbS42vVs9pKy0i9phyPOfUpTSoaQQRm/h02toWKwTFPAEBS21ZUELZmkAKJFnjzmYL2Sy7v/PHNUPuXWaS2d2Zve5r5vt+ve7X7H3Nfe/928nkN9dc9/Vg7o6IiKRrRuwARERkapTIRUQSp0QuIpI4JXIRkcQpkYuIJE6JXEQkcUrkIiKJUyIXEUmcErmISOKUyEVEEqdELiKSOCVyEZHEKZGLiCROiVySYWZvNjM3s0vLPHd58bk3nuD8WWb2CTP7VzM7amY/MbNdZrYqc8zNZjZoZq8ws9vN7D/M7Akzu9rMbNzv+x9mdnfx9zxjZveb2TsrXPt9ZnafmT1tZj81sx1mdtG4Y95qZjuL1zxiZt8ys1dP/JWSZqNELinZBjwOvLfMc+8Fet39nhOcfw3wCeB7wDpgPfAY8IZxx7UAtwEHgN8H7gc+Wdyyfgf4AXA18IfAs8A3zOyXsgeZ2SeALwPDxWM/Ufw7VmaO+VXgW8Ag8AfA/wR+DthlZmee4G8SAXfXpi2ZDfgT4CjQnik7jZAkrznJuf8M3HqSY24GHLg+U2bArcAQ0JEpP2XcubOAHuC7mbJXAiPAXwMzxh1vxce5wE+Bvxj3/Hygf3y5Nm3jN9XIJTW3AK1AtgnjMmAm8JcnObcfeLWZvaqK62wo/eDuXtyfDVyYKX+m9LOZvRhoB3YytoZ/CeGb7x+7+2j2AsXfC7AKeBHwFTPrKG2ED4B7gTdXEa80sZmxAxCZCHd/2My+T2hKubFY/F5gt7s/epLTrwb+FnjEzB4kNJ982d0fGHfcKPBv48oeKT6eWSows7cBVwGvI3y4PBdm5ufO4u/74QniKn2w3Fnh+YETnCuiRC5JugX4MzNbSEig5xHavE/I3XeYWSdwMXAR8OvAR8zst9z9CxMJwMz+C/B3wA7gt4EnCc077wcun8jv4vi9ql8Fnirz/LMT/H3SZJTIJUVfBT4D/ApwCiGBfq2aE939MPBF4ItmNpeQiK8Bsol8BvAKjtfCAc4uPu4tPr6D0Fa/2t2HSgeZ2fvHXbK3+Pt+jtBGX05v8fGgu3+nmr9DJEtt5JIcd+8Dvg28j9Csclux7ITM7CXjfs8g8Chjm0VKnqvhF7sdriN8YHy3WDxCaEJpyRx3JqFNPOubhKaVq81szP+3THfG2wnNJ39oZrPKxH3aCf8waXqqkUuqbgH+X/Hnj1d5zg/NbBuhO+FhYCnhpumGcccdBX7BzL5EuNn4VuCXgD9x90PFY74FfBS4zcy2AC8FPkT4YDin9Ivc/VEzW1+McaeZ/TWh98sy4AngSncfMLMrCF0U/8nMvgocAs4oXvcuqmg6kuZlx2+ci6TDzGYT2pNnAAvc/WgV5/wR8MuEZpJWYB8hef4fdx8uHnMzIbmfA3wOeBNwBPg843qemNla4GOEhPtj4NOEm6GfcPfxg4feD3yY0MTyNPAA8KlsU4qZXVD8fecV4/t3Qi+YDe5+f5UvjTQhJXJJkpnNJNRo/97dP1DD33sz8E53n1ur3ylSb2ojl1RdQhgIdEvkOESiUxu5JMXMziU0e3wc+IG7b48ckkh0qpFLaq4gtF0fBP5b5FhEckFt5CIiiVONXEQkcUrkIiKJS/pmZ3Fk3MsI/XxFRBrFPOAJr7LtO+lETkji+2MHISJSBwsJg8JOKvVEfgTg8ccfp62tLXYsIiJTNjAwwMtf/nKYQEtD6okcgLa2NiVyEWlautkpIpI4JXIRkcQpkYuIJE6JXEQkcUrkIiKJUyIXaUaaY6mhKJGLNJvREbhtaXiUhqBELtJsDu2EQg8c2hU7EqmRhhgQJCIncbQPhgvh54e7YXQY9nTDCxaGslntMKcjXnwyJUrkIs3gofXwyAaYfSqMHgtlB7bD1i44dhjOXgdLuuPGKJOmphWRZrCkG5ZvBGuB4f5QNtwPNhOWb1IST1zURG5me83My2w3xIxLpCF1roX5K8eWLVgJnWuihCO1E7tpZRnQktl/DXAH8I044Yg0MHc4uB1aO2BuJwz2woFtodwsdnQyBVFr5O5+yN2fKm3A24BeQCuji9Rafw88OwhdW2D1bujaHPb7e2JHJlOUm8WXzWw28ATwGXf/kwrHtAKtmaJ5wP5CoaBpbEVOZmQIRodgVub/yrECtMyBltbK58m0GhgYoL29HaDd3QeqOSdPNzsvAV4E3HyCY64ECplNqwOJVKuldWwSB5jdriTeAPKUyD8AfNvdnzjBMdcC7Zlt4XQEJiKSZ7FvdgJgZj8LXAi8/UTHufsQMJQ5r86RiYjkX15q5O8HDgLfih2IiEhqoidyM5tBSORfcvdnY8cjIpKa6Imc0KRyBnBT7EBERFIUvY3c3bcCauwWEZmkPNTIRURkCpTIRUQSp0QuIpI4JXIRkcQpkYuIJE6JXEQkcUrkIiKJUyIXEUmcErmISOKUyEVEEqdELiKSOCVyEZHEKZGLiCROiVxEJHFK5CIiiVMiF5HK3GNHIFVQIheR8kZGYOnS8Ci5pkQuIuXt3Ak9PbBrV+xI5CSiL/UmIjnS1weFQvi5uxuGh8PjwoWhrL0dOjrixSdlqUYuIsetXw+LF0NXF+zYEcq2bw/7ixeH5yV3lMhF5Ljubti4EVpaoL8/lPX3w8yZsGlTeF5yR4lcRMZauxZWrhxbtnIlrFkTJRw5ObWRi8hY7qE5paMDOjuhtxe2bQvlZrGjkzJUIxeRsXp6YHAQtmyB3bth8+aw39MTOzKpwDzhDv9m1gYUCoUCbW1tscMRaQxDQ2HL/p8qFGDOHGhtjRdXkxgYGKC9vR2g3d0HqjlHTSsiMlZr6/MTdkgsklNqWhERSVz0RG5mP2Nmf2lmPzGzZ8ysx8yWxo5LRCQVUZtWzOzFwF3A94C3AoeAVwE/jRmXiEhKYreR/wHwuLu/P1P241jBiIikKHbTyi8D/2hm3zCzg2b2AzP7jUoHm1mrmbWVNmDe9IUqIpJPsRP5K4ArgH8FVgOfA643s1+rcPyVQCGz7Z+OIEVE8ixqP3IzOwb8o7t3ZcquB5a5+xvLHN8KZPtFzQP2qx+5iDSKFPuRPwn8cFzZj4B3lDvY3YeAodK+abiwiEj0ppW7gEXjys4G9kWIRUQkSbETeTdwnpn9oZm90swuB34TuCFyXCIiyYiayN39+8ClwK8ADwIfB37X3TfHjEtEJCWx28hx91uBW2PHISKSqthNKyIiMkVK5CIiiVMiFxFJnBK5NKeEF1QRGU+JXJrP6AjctjQ8ijQAJXJpPod2QqEHDu2KHYlITUTvfigyLY72wXAh/PxwN4wOw55ueMHCUDarHeZ0xItPZAqUyKU5PLQeHtkAs0+F0WOh7MB22NoFxw7D2etgSXfcGEUmSU0r0hyWdMPyjWAtMNwfyob7wWbC8k1K4pI0JXJpHp1rYf7KsWULVkLnmijhiNSKmlakebjDwe3Q2gFzO2GwFw5sC+WaElkSphq5NI/+Hnh2ELq2wOrd0LU57Pf3xI5MZEqirhA0VcV1OwtaIUiqMjIEo0MwK/NeOVaAljnQ0lr5PJFplOIKQSLTp6X1+Ql7dnucWERqSE0rIiKJUyIXEUmcErmISOKUyEVEEqdELiKSOCVyEZHEKZGLiCROiVxEJHFK5CIiiVMiFxFJnBK5iEjilMhFRBKnRC4ikjglchGRxEVN5GZ2jZn5uO3hmDGJiKQmD/ORPwRcmNl/NlYgIiIpykMif9bdn4odhIhIqvLQRv4qM3vCzP7NzDab2RmVDjSzVjNrK23AvGmMU0Qkl2In8nuBNcAvAFcAZwE7zaxSgr4SKGS2/dMQo4hIruVq8WUzexGwD/iou99Y5vlWILvo4jxgvxZfFpFGkfziy+7eb2aPAK+s8PwQMFTaN7PpCk1EJLdiN62MYWZzgU7gydixiIikInY/8v9rZivM7Ewz6wL+BhgBvhIzLhGRlMRuWllISNovAQ4Bu4Dz3P1Q1KhERBISNZG7+3tiXl9EpBHkqo1cREQmTolcRCRxSuQiIolTIhcRSZwSuYhI4pTIRUQSp0QuIpI4JXIRkcQpkYuIJE6JXEQkcUrkUjs5mttepJkokUttjI7AbUvDY6PSB5XklBK51MahnVDogUO7YkdSHyMjsHRpeBTJGSVymbyjfXCkN2wPd8PoMOzpPl52tC92hLWzcyf09MCuBv2gkqTFno9cUvbQenhkA8w+FUaPhbID22FrFxw7DGevgyXdcWOcir4+KBTCz93dMDwcHhcuDGXt7dDRES8+kSLVyGXylnTD8o1gLTDcH8qG+8FmwvJNaSdxgPXrYfFi6OqCHTtC2fbtYX/x4vC8SA4okcvUdK6F+SvHli1YCZ1rooRTU93dsHEjtLRAf38o6++HmTNh06bwvEgOKJHL1LjDwe3Q2gEvOTc8HtjWOD081q6FleM+qFauhDVrooQjUo4SuUxNfw88OwhdW2D1bujaHPb7e2JHVhvuoTmlowPOPTc8btvWOB9U0hCUyGVq2hbBJfvg9FVh//SL4OK9obwR9PTA4CBs2QK7d8PmzWG/p0E+qKQhmCdcszCzNqBQKBRoa2uLHY40oqGhsGXfX4UCzJkDra3x4pKGNTAwQHt7O0C7uw9Uc466H4qcSGvr8xN2+E8mkhtqWhERSZwSuYhI4pTIRUQSp0QuIpI4JXIRkcQpkYuIJC43idzMPmZmbmbXxY5FRCQluUjkZrYM+CDwQOxYRERSEz2Rm9lcYDPwG8BPI4cjIpKc6IkcuAHLine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAGDCAYAAAA26pu1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAv+klEQVR4nO3de3xU9Z3/8deHXMGQoAa5LFIwKNQLq+VmowJSENvtr2i1YrXtUrq7rUqrrb+ttXWR+itS29+a1tJaFi9oK0r72620dKuoNBEUxVrRVASVAuqKkEgTiEASku/vj+9MmIQM5DIz55zk/Xw8zmNyvvOdOZ9MJp858z3fiznnEBGR6OoTdAAiItI9SuQiIhGnRC4iEnFK5CIiEadELiIScUrkIiIRp0QuIhJxSuQiIhGnRC4iEnFK5CIiEadELiIScUrkIiIRp0QuIhJxSuQSGWZ2oZk5M7u0nfuuit330aM8PsfMbjWzN8zsoJm9b2brzGxGQp1lZlZnZqeY2eNm9oGZvWtm883M2jzf/zazZ2PPc8DMXjSzy5Mc+3NmtsHM9pvZ38zsaTO7qE2dj5vZ2tgx95nZ783sjM6/UtLbKJFLlJQDbwNXt3Pf1cBW59z6ozx+AXAr8EdgHrAQeAv4SJt6WcBjwC7gm8CLwHdjW6LrgZeA+cC3gUPAr83sHxIrmdmtwC+AxljdW2O/x7SEOp8Hfg/UATcB/wc4HVhnZiOO8juJgHNOm7bIbMDtwEGgKKFsID5JLjjGYzcCq45RZxnggLsSygxYBdQDxQnlfds8NgeoBJ5KKBsFNAH/BfRpU99itwXA34D/aHP/IKCmbbk2bW03nZFL1DwI5AGJTRizgWzgl8d4bA1whpmd2oHjLI7/4Jxzsf1cYHpC+YH4z2Z2PFAErKX1Gf4l+G++tznnmhMPEHtegBnAAOBhMyuOb/gPgOeBCzsQr/Ri2UEHINIZzrnNZvYCvinl3ljx1cBzzrk3j/Hw+cBK4HUz+wu++eQXzrlX2tRrBv7apuz12O2IeIGZfRK4BTgb/+HSEmbCzyWx59t0lLjiHyxrkty/9yiPFVEil0h6EPixmQ3DJ9Bz8W3eR+Wce9rMSoBZwEXAPwFfN7OvOOfu6UwAZnYB8FvgaeBaYCe+eeeLwFWdeS4OX6v6PPBeO/cf6uTzSS+jRC5R9AhwJ/BZoC8+ga7oyAOdc3uA+4H7zawAn4gXAImJvA9wCofPwgFOi91uj91ehm+rn+mcq49XMrMvtjnk1tjznY5vo2/P1tjtbufckx35PUQSqY1cIsc5Vw38AfgcvlnlsVjZUZnZiW2epw54k9bNInEtZ/ixbofz8B8YT8WKm/BNKFkJ9Ubg28QTPYpvWplvZq3+3xK6Mz6Obz75tpnltBP3wKP+YtLr6YxcoupB4P/Ffv63Dj5mk5mV47sT7gHG4y+aLm5T7yBwsZk9gL/Y+HHgH4DbnXNVsTq/B74BPGZmy4GTgOvwHwxj40/knHvTzBbGYlxrZv+F7/0yAXgXuNk5t9fMrsF3UfyzmT0CVAHDY8d9hg40HUnvZYcvnItEh5nl4tuT+wCDnXMHO/CY7wCfwjeT5AE78Mnzh865xlidZfjkPha4Gzgf2Af8nDY9T8xsLvAtfMLdBtyBvxh6q3Ou7eChLwJfxTex7AdeAb6X2JRiZlNjz3duLL7/wfeCWeyce7GDL430QkrkEklmlo0/o/2dc+5LKXzeZcDlzrmCVD2nSLqpjVyi6hL8QKAHA45DJHBqI5dIMbNJ+GaPfwNecs5VBBySSOB0Ri5Rcw2+7Xo38IWAYxEJBbWRi4hEnM7IRUQiTolcRCTiIn2xMzYybii+n6+ISE/RH3jXdbDtO9KJHJ/E3wk6CBGRNBiGHxR2TFFP5PsA3n77bQoLC4OORUSk2/bu3cvJJ58MnWhpiHoiB6CwsFCJXER6LV3sFBGJOCVyEZGIUyIXEYm4HtFGfjTOOQ4dOkRTU1PQoQiQlZVFdnY2h9dUEJHu6tGJvKGhgZ07d7J///6gQ5EE/fr1Y8iQIeTm5gYdikiP0GMTeXNzM9u2bSMrK4uhQ4eSm5urs8CAOedoaGigqqqKbdu2ceqpp9Knj1r3AuEc6P+hx+ixibyhoYHm5mZOPvlk+vXrF3Q4EtO3b19ycnLYsWMHDQ0N5OfnBx1S79PcBI9PhJkboE/WsetL6PX40yGd8YWP/iYBq1oLtZVQtS7oSCRFeuwZuYgkOFgNjbX+581l0NwIW8qg3zBfllME+cXBxSfdokQu0hu8uhBeXwy5J0Bzgy/bVQGrS6FhD5w2D8aVBRujdJm+44r0BuPKYOISsCxorPFljTVg2TBxqZJ4xAWayM1su5m5drafBhlXGLz33ntcf/31jBo1ivz8fAYNGsR5553H3Xff3dKdcsSIEZgZZkbfvn0ZMWIEV1xxBWvWrAk4egmlkrkwaFrrssHToGROIOFI6gR9Rj4BGJKwzYiV/zqwiI4iU6vi/fWvf+Wcc85h9erV3H777bz00kusX7+eb37zm6xatYonn3yype5tt93Gzp072bJlCw8++CADBgxg+vTpLFy4MDPBSnQ4B7srIK8YTpzkb3eVZ+6NLWkTaBu5c64qcd/MvgVsBUK3MnpTE0ycCBs2QFaae2xde+21ZGdn86c//YnjjjuupfyUU05h1qxZJM41379/fwYPHgzA8OHDmTx5MkOGDGH+/PlcfvnljB49Or3BSnTUVMKhOjj/VzBkBuxcDetm+/LjxwYdnXRD0GfkLcwsF/gccF+yVTHMLM/MCuMbfhWNjFi7FiorYV2ae2y9//77rF69muuuu65VEk90rIFN119/Pc45Vq5cmY4QJaoKR8MlO3wSBxhyEcza7ssl0kKTyIFLgAHAsqPUuRmoTdjSujpQdTVs3eq3sjJobPS38bLq6tQf880338Q5d8SZdHFxMQUFBRQUFHDTTTcd9TlOOOEETjrpJLZv3576ACW6svIgp828/blFvlwiLUyJ/EvAH5xz7x6lziKgKGEbls6AFi6EMWOgtBSeftqXVVT4/TFj/P2ZsmHDBjZu3MgZZ5xBfX39Mes75zQlgUgvEYp+5Gb2IWA68Omj1XPO1QMtWSzdiaqsDM46C265BWpqfFlNDQwdCkuXwpw5qT/mqFGjMDO2bNnSqvyUU04B/BD3Y3n//fepqqpi5MiRqQ9QREInLGfkXwR2A78POpC25s6FaW16bE2blp4kDnDiiScyY8YMFi9ezAcffNCl5/jxj39Mnz59uOSSS1IbnIiEUuBn5GbWB5/IH3DOHQo6nrac880pxcVQUuLbxsvL0zt53M9+9jPOO+88xo8fz4IFCxg7dix9+vThhRdeYPPmzYwbN66l7r59+3jvvfdobGxk27Zt/PKXv+See+5h0aJFjBo1Kj0Biki4OOcC3YCLAAec1oXHFgKutrbWtXXgwAG3adMmd+DAgSPu64yXX3ZuwADnVq/2+48/7vdffrlbT3tM7777rps3b54bOXKky8nJcQUFBW7ixInuhz/8ofvggw+cc8596EMfcrHXzuXm5rrhw4e7K664wq1Zsya9wXVTqv42Ij1RbW1t/P+60HUwF5qL8GCAWBfE2traWgoLW1+NP3jwINu2bWPkyJHdmiq1vt5viU9fWwv5+ZCni/1dkqq/jUhPtHfvXoqKigCKnHN7O/KYwJtWwi4v78iE7V9jEZFwCMvFThER6SIlchGRiFMiFxGJOCVyEZGIUyIXEYk4JXIRkYhTIhcRiTglchGRiFMiFxGJOCXyEJozZw5mxve///1W5Y8++mjapu5dsGABZ599dlqeW0TSS4m8MzI4L01+fj533HEHf/vb3zJ2TBGJJiXyjmpugsfG+9sMmD59OoMHD2bRokVJ66xbt44LLriAvn37cvLJJ/O1r32tZQ7zxYsXc+aZZ7bUjZ/N//znP291jFtuuYVly5bx3e9+l5dffhkzw8xYtmwZAG+99RazZs2ioKCAwsJCrrjiCnbt2tXyHPEz+V/84heMGDGCoqIirrzySvbt25fiV0REklEi76iqtVBbCVVpXn05Jisri9tvv52f/OQnvPPOkUuTbt26lYsvvpjLLruMV155hRUrVrBu3TrmzZsHwJQpU9i0aRNVVVUAVFRUUFxcTHl5OQCNjY2sX7+eqVOnMnv2bG688UbOOOMMdu7cyc6dO5k9ezbNzc3MmjWLPXv2UFFRwRNPPMFf//pXZs+efUQsjz76KKtWrWLVqlVUVFQc0SwkIumjRH40B6th31a/bS6D5kbYUna47GAaVl9OcOmll3L22Wdz6623HnHfokWLuPrqq7nhhhs49dRTKS0t5a677uLBBx/k4MGDnHnmmZxwwglUVFQAUF5ezo033tiyv2HDBhobGyktLaVv374UFBSQnZ3N4MGDGTx4MH379uWpp56isrKS5cuXM27cOCZNmsSDDz5IRUUFL7zwQksszc3NLFu2jDPPPJMLLriAz3/+8zz11FNpfW1E5DAl8qN5dSGsGgOrS2F3bPXlXRV+f9UYf3+a3XHHHTzwwAO89tprrcpffvllli1bRkFBQcs2c+ZMmpub2bZtG2bG5MmTKS8vp6amhk2bNnHttddSX1/P5s2bqaioYMKECfTr1y/psV977TVOPvlkTj755Jay008/nQEDBrSKZ8SIEfTv379lf8iQIezevTuFr4IEJsLrFfQmSuRHM64MJi4By4LGGl/WWAOWDROX+vvTbPLkycycOZObb765VXldXR1f/vKX2bhxY8v28ssv88Ybb1BSUgLA1KlTKS8vZ+3atZxzzjkUFha2JPeKigqmTJmSkhhzcnJa7ZsZzc3NKXluCVBTE4wf728l1JTIj6VkLgxqs/ry4GlQMidjIXz/+9/nd7/7HevXr28p+8hHPsKmTZsYNWrUEVtubi5wuJ3817/+NVOnTgV8cn/yySd55plnWsoAcnNzaWrLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 400x400 with 1 Axes>"
|
||||
]
|
||||
@@ -748,12 +742,11 @@
|
||||
"source": [
|
||||
"axes = plt.figure(figsize=(4, 4), dpi=100).gca()\n",
|
||||
"axes.set_title('y space')\n",
|
||||
"axes.scatter(historyGDy[:,0], historyGDy[:,1], lw=0.5, marker='*', color='blue')\n",
|
||||
"axes.scatter(historyNty[:,0], historyNty[:,1], lw=0.5, marker='*', color='orange')\n",
|
||||
"axes.scatter(historyPGy[:,0], historyPGy[:,1], lw=0.5, marker='*', color='red')\n",
|
||||
"axes.scatter(historyGDy[:,0], historyGDy[:,1], lw=0.5, marker='*', color='blue', label='GD')\n",
|
||||
"axes.scatter(historyNty[:,0], historyNty[:,1], lw=0.5, marker='*', color='orange', label='Newton')\n",
|
||||
"axes.scatter(historyPGy[:,0], historyPGy[:,1], lw=0.5, marker='*', color='red', label='PG')\n",
|
||||
"axes.scatter([0], [0], lw=0.25, color='black', marker='*') \n",
|
||||
"axes.set_xlabel('z0')\n",
|
||||
"axes.set_ylabel('z1')"
|
||||
"axes.set_xlabel('z0'); axes.set_ylabel('z1'); axes.legend()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -771,13 +764,13 @@
|
||||
"source": [
|
||||
"## Conclusions \n",
|
||||
"\n",
|
||||
"Despite its simplicity, this simple example already shows surprisingly large differences between gradient descent, Newton's method, and using the _inverse simulator_.\n",
|
||||
"Despite its simplicity, this example already shows surprisingly large differences between gradient descent, Newton's method, and using the _inverse simulator_.\n",
|
||||
"\n",
|
||||
"The main takeaways of this section are the following.\n",
|
||||
"* GD easily yields \"unbalanced\" updates, and gets stuck.\n",
|
||||
"* Newtons method does better, but is far from optimal.\n",
|
||||
"* the higher-order information of the invese simulator outperform both, even if it is applied only partially (we still used Newton's method for $L$ above).\n",
|
||||
"* Also, the choice of optimizer strongly affects progress in latent spaces like $\\mathbf{y}$.\n",
|
||||
"* Also, the methods (and in general the choice of optimizer) strongly affects progress in latent spaces, as shown for $\\mathbf{y}$ above.\n",
|
||||
" \n",
|
||||
"In the next sections we can build on these observations to use PGs for training NNs via invertible physical models."
|
||||
]
|
||||
@@ -791,7 +784,7 @@
|
||||
"\n",
|
||||
"## Approximate inversions\n",
|
||||
"\n",
|
||||
"If an analytic inverse like the `fun_y_inv_analytic` above is not readily available, we can actually resort to optimization schemes like Newton's method or BFGS to obtain a local inverse numerically. This is a topic that is orthogonal to the comparison of different optimization methods, but it can be easily illustrated based on the PG example above.\n",
|
||||
"If an analytic inverse like the `fun_y_inv_analytic` above is not readily available, we can actually resort to optimization schemes like Newton's method or BFGS to obtain a local inverse numerically. This is a topic that is orthogonal to the comparison of different optimization methods, but it can be easily illustrated based on the inverse simulator variant from above.\n",
|
||||
"\n",
|
||||
"Below, we'll use the BFGS variant `fmin_l_bfgs_b` from `scipy` to compute the inverse. It's not very complicated, but we'll use numpy and scipy directly here, which makes the code a bit messier than it should be."
|
||||
]
|
||||
@@ -844,7 +837,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Nonetheless, we can now use this numerically inverted $\\mathbf{y}$ function to perform the PG optimization. Apart from calling `fun_y_inv_opt`, the rest of the code is unchanged."
|
||||
"Nonetheless, we can now use this numerically inverted $\\mathbf{y}$ function to perform the inverse simulator optimization. Apart from calling `fun_y_inv_opt`, the rest of the code is unchanged."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -891,7 +884,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Nice! It works, just like PG. Not much point plotting this, it's basically the PG version, but let's measure the difference. Below, we compute the MAE, which for this simple example turns out to be on the order of our floating point accuracy."
|
||||
"Nice! It works, just like the PG version above. Not much point plotting this, it's basically the same, but let's measure the difference. Below, we compute the MAE, which for this simple example turns out to be on the order of our floating point accuracy."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -930,13 +923,6 @@
|
||||
"\n",
|
||||
"- Replace the simple \"regular\" gradient descent with another optimizer, e.g., commonly used DL optimizers such as AdaGrad, RmsProp or Adam. Compare the versions above with the new trajectories."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
@@ -960,4 +946,4 @@
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user