daviddoji/pbdl-book
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

updated HIG notebook

Browse Source
This commit is contained in:
NT committed 2022-04-21 17:31:59 +02:00
1 parent a8c116eb1e
commit 6427365d55
2 files changed
+227 -229
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

json-cleanup-for-pdf.py
+1
View File
@@ -58,6 +58,7 @@ for fnOut in fileList:
res.append( re.compile(r"ERROR: pip") ) # pip dependencies res.append( re.compile(r"ERROR: pip") ) # pip dependencies
res.append( re.compile(r"requires imgaug") ) # pip dependencies res.append( re.compile(r"requires imgaug") ) # pip dependencies
res.append( re.compile(r"See the documentation of nn.Upsample") ) # pip dependencies
# remove all "warnings.warn" from phiflow? # remove all "warnings.warn" from phiflow?
physgrad-code.ipynb
+226 -229
View File
@@ -2,6 +2,10 @@
"cells": [ "cells": [
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"collapsed": false,
"id": "XLjsvhkuS2eJ"
},
"source": [ "source": [
"# Learning to Invert Heat Conduction with Scale-invariant Updates\n", "# Learning to Invert Heat Conduction with Scale-invariant Updates\n",
"\n", "\n",
@@ -11,14 +15,13 @@
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/physgrad-code.ipynb)\n", "[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/physgrad-code.ipynb)\n",
"\n", "\n",
"\n" "\n"
], ]
"metadata": {
"collapsed": false,
"id": "XLjsvhkuS2eJ"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "9fETS9hwrQrc"
},
"source": [ "source": [
"## Problem Statement\n", "## Problem Statement\n",
"\n", "\n",
@@ -34,13 +37,13 @@
"\n", "\n",
"$$L(x) = || \\mathcal P(x) - y^* ||_2^2 = || \\mathcal F^{-1}\\left( \\mathcal F(x) \\cdot e^{-k^2 (t_* - t_0)} \\right) - y^* ||_2^2 . $$\n", "$$L(x) = || \\mathcal P(x) - y^* ||_2^2 = || \\mathcal F^{-1}\\left( \\mathcal F(x) \\cdot e^{-k^2 (t_* - t_0)} \\right) - y^* ||_2^2 . $$\n",
"\n" "\n"
], ]
"metadata": {
"id": "9fETS9hwrQrc"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "EA7qc5mXsZLU"
},
"source": [ "source": [
"---\n", "---\n",
"\n", "\n",
@@ -51,43 +54,46 @@
"\n", "\n",
"We'll use phiflow with PyTorch as default backend, but this example code likewise runs with TensorFlow (just switch to `phi.tf.flow` below).\n", "We'll use phiflow with PyTorch as default backend, but this example code likewise runs with TensorFlow (just switch to `phi.tf.flow` below).\n",
"\n" "\n"
], ]
"metadata": {
"id": "EA7qc5mXsZLU"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 17, "execution_count": null,
"metadata": {
"id": "vN06frDqS2eM",
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"!pip install --upgrade --quiet git+https://github.com/tum-pbs/PhiFlow\n", "!pip install --upgrade --quiet git+https://github.com/tum-pbs/PhiFlow\n",
"from phi.torch.flow import * # switch to TF with \"phi.tf.flow\"" "from phi.torch.flow import * # switch to TF with \"phi.tf.flow\""
], ]
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "vN06frDqS2eM"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"collapsed": false,
"id": "fc6Am-XcS2eN"
},
"source": [ "source": [
"\n", "\n",
"## Data generation\n", "## Data generation\n",
"\n", "\n",
"For training, we generate $x^*$ by randomly placing between 4 and 10 \"hot\" rectangles of random size and shape in the domain. The `generate_heat_example()` function below generates a full mini batch (via `shape.batch`) of example positions. These will be $x$ later on. They're never passed to the solver, but should be reconstructed by the neural network.\n", "For training, we generate $x^*$ by randomly placing between 4 and 10 \"hot\" rectangles of random size and shape in the domain. The `generate_heat_example()` function below generates a full mini batch (via `shape.batch`) of example positions. These will be $x$ later on. They're never passed to the solver, but should be reconstructed by the neural network.\n",
"\n" "\n"
], ]
"metadata": {
"collapsed": false,
"id": "fc6Am-XcS2eN"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 18, "execution_count": null,
"metadata": {
"id": "-39wQc1ES2eN",
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"def generate_heat_example(*shape, bounds: Box = None):\n", "def generate_heat_example(*shape, bounds: Box = None):\n",
@@ -106,58 +112,56 @@
" component_mask = component_mask.at(heat_t0)\n", " component_mask = component_mask.at(heat_t0)\n",
" heat_t0 += component_mask * strength\n", " heat_t0 += component_mask * strength\n",
" return heat_t0" " return heat_t0"
], ]
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "-39wQc1ES2eN"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"source": [
"The data is generated on-the-fly later during training, but let's look at two example $x$ for now using phiflow's `vis.plot` function:"
],
"metadata": { "metadata": {
"collapsed": false, "collapsed": false,
"id": "sz_eZyZ7S2eO" "id": "sz_eZyZ7S2eO"
} },
"source": [
"The data is generated on-the-fly later during training, but let's look at two example $x$ for now using phiflow's `vis.plot` function:"
]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 19, "execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 864x360 with 4 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAz4AAAFgCAYAAAB60Kx2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df7BkdXnn8fczM8DID+XHxAlhMExWjGGjgDshEq0NSsyOrgXWxqJkEzOmyE5lK6Z0k2zEbFZ3SbIbk90YU0WymVUiSamEJTFMJSRIENckKjIERJgRHVHCIDiCYFiNDHPvs3/0GXOZOz/udPe3Tz9936+qrtvn2327n4au+5nnnO/5nshMJEmSJGmWrei7AEmSJElqzcZHkiRJ0syz8ZEkSZI082x8JEmSJM08Gx9JkiRJM29V3wUsxdFxTK7muL7LkKSRfJOvsyefjHG+5r962XH56Ffnhvrd2+968sbM3DjOemaN+SNpFow7f0bJHugvf0o0Pqs5ju+PC/suQ5JGcmvePPbXfPSrc3zyxucM9bsrT/3cmjGXM3PMH0mzYNz5M0r2QH/5U6LxkSQdWALzzPddhiRpGamaPTY+klRaMpf1wkeSVFnN7HFxA0mSJEkzzyM+klTYYLpB9l2GJGkZqZo9Nj6SVFzFedaSpNoqZo+NjyQVliRzWW+vmySprqrZY+MjScVVnG4gSaqtYvbY+EhSYQnMFQwfSVJdVbPHVd0kSQcUEVdFxO6IuPsgj/9oRNwVEZ+OiI9FxNmTrlGSNHta5Y+NjyQVN08OdVuC9wIbD/H4F4AfzMwXAL8MbBn900iSKhg2e/rMH6e6SVJhCc1OMM3Mj0bEGYd4/GMLNj8BrGtSiCRpqrTMHmiXPzY+klTcCAuKromIbQu2t2TmsEdtLgP+YvhSJEmVjLiYdS/5Y+MjSYUlOcoJpo9k5oZRa4iIlzEInpeO+lqSpOk3YvZAT/lj4yNJlSXM9biwTkS8EHg38MrMfLS/SiRJE9Nz9sBw+dN0cYOIODEirouIz0TEjog4PyJOjoibIuJz3c+TWtYgSWojIp4D/Anw+sz8bN/1LGT+SNLsGjZ/Wq/q9i7gLzPz+cDZwA7gcuDmzDwTuLnbliQNIRnMsx7mdjgR8QHg48B3R8SuiLgsIn4qIn6qe8rbgFOA34mIO/ebr90380eSGhkle/rMn2ZT3SLiWcC/BN4AkJl7gD0RcTFwQfe0q4GPAG9pVYckzbZgjmjyypl56WEe/0ngJ5u8+QjMH0lqrV32QLv8aXnEZz3wFeD3I+KOiHh3RBwHrM3Mh7rnPAysbViDJM20BOZzuNsMM38kqaFRsqfP/GnZ+KwCXgT8bmaeC3yd/aYVZGbCgZeEiIjNEbEtIrY9xZMNy5Sk2ua6PW9Hepth5o8kNTZs9vSZPy0bn13Arsy8tdu+jkEQfTkiTgXofu4+0C9n5pbM3JCZG47imIZlSlJdiY3PAZg/ktTQKNkzk41PZj4MPBAR390NXQhsB7YCm7qxTcD1rWqQJC0/5o8k6UBaX8fnZ4D3RcTRwH3ATzBotq6NiMuA+4FLGtcgSTNtPmf66M2wzB9Jaqhi9jRtfDLzTuBAV2W9sOX7StJysW+6gZ7O/JGkdqpmT+sjPpKkhpJgrvkl2SRJ+idVs8fGR5KKqzjdQJJUW8XssfGRpMKqTjeQJNVVNXvqHaOSJEmSpCPkER9JKi2YS/dhSZImqWb22PhIUmEJzHvwXpI0QVWzx8ZHkoqrOM9aklRbxeyx8ZGkwjJrTjeQJNVVNXtsfCSpuPmCe90kSbVVzB4bHw0n6n3Zxyaz7wokSZJ0hGx8JKmwwbUU6k03kCTVVTV7bHwkqbSa86wlSZXVzB4bH0kqrOqSopKkuqpmj42PJBU3l8v4nDtJUi8qZo+NjyQVlkTJedaqZ8UJJywai9Wrh369ua98ZZRyJPWoavbUq1iSJEmSjpBHfCSpuPmCJ5hKkmqrmD02PpJUWNUlRSVJdVXNHhsfSSosiZInmGo2PPnC5ywamztm8T+GnvGlry/+Zc/xkcqqmj02PpJUXMUlRSVJtVXMHhsfSSosk5IXkZMk1VU1e+pVLEmSJElHyCM+klRaME+9edaSpMpqZo+NjyQVltScbqDZlSvq/WNI0pGpmj02PpJUXMUlRSVJtVXMHhsfSSosCeYLLikqSaqravbY+EhScRX3ukmSaquYPfUqliRJkqQj5BEfSSosgfmCJ5hKkuqqmj02PpJUWjBXcElRSVJlNbPHxkeSCqu6102SVFfV7LHxkaTiKu51kyTVVjF7bHwkqbDMKLnXTZJUV9XssfHRYT3++vP7LgGAE//w432XIEmSpKLqtWqSpKeZyxVD3Q4nIq6KiN0RcfdBHo+I+O2I2BkRd0XEi8b+4SRJU2nY7Okzf2x8JKmwBOaJoW5L8F5g4yEefyVwZnfbDPzuqJ9HkjT9RsmePvPHqW6SVFosae/ZMDLzoxFxxiGecjHwB5mZwCci4sSIODUzH2pSkCRpSrTLHmiXPzY+klTYYEnRoVfWWRMR2xZsb8nMLUfw+6cBDyzY3tWN2fhI0gwbMXugp/yx8dFh7Xnm4i/2/Csea/qeq/7ixKavL82SueFnLT+SmRvGWYskaXkYIXugp/xp2vhExBeBJ4A5YG9mboiIk4E/As4Avghckplt/xUtSWrhQeD0BdvrurHemT+SNNOGyp9JLG7wssw8Z0FXdzlwc2aeCdzcbUuShpAE8zncbQy2Aj/era7zYuBrU3Z+j/kjSQ2Mkj195k8fU90uBi7o7l8NfAR4Sw91SNJMmG+0DysiPsDg7/WaiNgFvB04CiAz/xdwA/AqYCfwDeAnmhQyPuaPJI1Jq+yBdvnTuvFJ4EMRkcDvdSctrV3QkT0MrG1cgyTNrEyYG8/eswO8dl56mMcT+Okmbz4680eSGmmZPYPXb5M/rRufl2bmgxHxbOCmiPjMwgczM7tQWiQiNjNYl5vVHNu4TEmqa0zTBmaN+SNJDVXMnqaNT2Y+2P3cHREfBM4Dvrxvne2IOBXYfZDf3QJsAXhmnHzAcJKk5W4wz9prUe/P/JGkdqpmT7OKI+K4iDhh333gh4G7GZyMtKl72ibg+lY1SJKWH/NHknQgLY/4rAU+GBH73uf9mfmXEXEbcG1EXAbcD1zSsAZJmnlz1Jtu0Jj5I0mNVcyeZo1PZt4HnH2A8UeBC1u9ryQtJ2O4evbMMX8kqa2q2dPHctaSpLGpOc9aklRZzeyx8ZGk4uYLTjeQJNVWMXtsfCSpsNbXUpAkaX9Vs6feMSpJkiRJOkIe8ZGk4irOs5Yk1VYxe2x8JKmwwUXk6k03kCTVVTV7bHwkqbiKJ5hKkmqrmD02PpJUWNVrKUiS6qqaPTY+klRcxXnWkqTaKmZPvYolSZIk6Qh5xEeSKsuaJ5hKkgormj02PpJUWFLzBFNJUl1Vs8fGR5KKq7jXTZJUW8XssfGRpMKqrqwjSaqravbY+EhScRXDR5JUW8XscVU3SZIkSTPPIz6SVFhSc2UdSVJdVbPHxkeSiqu4so4kqbaK2WPjI0mVZc151pKkwopmj42PJBVWdWUdSVJdVbPHxkeSiqsYPpKk2ipmj6u6SZIkSZp5HvGRpMKqrqwjSaqravbY+EhScVkwfCRJtVXMHhsfSSqu4pKikqTaKmaPjY8kFZZFlxSVJNVVNXtsfCSpuIrTDSRJtVXMHld1kyRJkjTzyh7xWfnc9YsHV8xeHzd/3/1P2869e3uqRNJ0qrmyjiSpsprZU7bxkSQNVJxuIEmqrWL22PhIUmFJzRNMJUl1Vc0eGx9JqiwHq+tIkjQxRbPHxkeSiqt4LQVJUm0Vs2emGp/HNnxb3yWMZPWjixcuOGa/xQ0kSZIkHbmZanwkablJap5gKkmqq2r2zN76z5K0rAyWFB3mtqRXj9gYEfdGxM6IuPwAjz8nIm6JiDsi4q6IeNXYP6IkacoMnz1LyZ9W2WPjI0nFZQ53O5yIWAlcCbwSOAu4NCLO2u9pvwRcm5nnAq8Dfme8n06SNI2GzZ7D5U/L7HGqmyQV13C6wXnAzsy8DyAirgEuBrYvfHvgmd39ZwFfalWMJGl6VMweGx9JKmyw92zo8FkTEdsWbG/JzC0Ltk8DHliwvQv4/v1e478AH4qInwGOA35o2GJUz+rtDw79u/NjrEPSZI2YPXDo/GmWPc0bn+5w1Tbgwcx8dUSsB64BTgFuB16fmXta1yFJWuSRzNww4mtcCrw3M/9nRJwP/GFEfG9m9v7vWvNHkqbWqPkzVPZM4hyfNwE7Fmy/A3hnZj4XeAy4bAI1SNLMari4wYPA6Qu213VjC10GXAuQmR8HVgNrxvCxxsH8kaRGGi5u0Cx7mjY+EbEO+NfAu7vtAF4OXNc95WrgNS1rkKRZ12pxA+A24MyIWB8RRzM4gXTrfs/5e+BCgIj4Hgbh85XxfbrhmD+S1FarxQ1omD2tp7r9FvALwAnd9inA45m570qduxjM45MkDanVCaaZuTci3gjcCKwErsrMeyLiCmBbZm4Ffg743xHxHxicbPqGzCW2VW2ZP5LUUMXsadb4RMSrgd2ZeXtEXDDE728GNgOs5tgxV6dRrbjppMbvMA3/bpKmXxJNLyKXmTcAN+w39rYF97cDL2lWwBDMnzbmn3hiSWPLxfxLz+m7hCVb8Td39l2CZkzV7Gl5xOclwEXdBYVWM1hy7l3AiRGxqtvrdqA5ewB0KztsAXhmnOy/giXpIPwDuYj5I0mNVfzj2Owcn8x8a2auy8wzGMzN+3Bm/ihwC/Da7mmbgOtb1SBJWn7MH0nSgUxiVbf9vQX42YjYyWDO9Xt6qEGSZkN3LYVhbsuQ+SNJ4zBC9vSZPxO5gGlmfgT4SHf/PgZXZFURz77yY32XIOlQKs43mBDzR63kqsX7jj+/qf8dCuvft3isj73cWLine truncated
},
"metadata": { "metadata": {
"needs_background": "light"
}
}
],
"source": [
"vis.plot(generate_heat_example(batch(view_examples=2), spatial(x=64, y=64)));"
],
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
"height": 390 "height": 390
}, },
"id": "BHtnPdL3S2eO", "id": "BHtnPdL3S2eO",
"outputId": "7937d111-1c65-44fb-e11a-1f219cc8ecb1" "outputId": "3dbf4cca-015b-4e9d-8274-acea430c6ecd",
"pycharm": {
"name": "#%%\n"
} }
}, },
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAz4AAAFgCAYAAAB60Kx2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df7BkdX3m8eeZGcbhhwjDKBkZFDaOMWyMSAhKtCKCSYasBdauZUmZhGRJ5o9Vy5TZJJjN6q5JdtdsJcZUYZIpJM6mooRgDLMukSDiuokRGSPya0RGhGVYYISBoCgDc+9n/+gz5M70nbn3dvfp00/f96uqa+75dt/uT0NPP/M553u+x1UlAAAAAJhmK7ouAAAAAADaRuMDAAAAYOrR+AAAAACYejQ+AAAAAKYejQ8AAACAqbeq6wIWY7WfU2t0dNdlAMBQntKTerr2epTP+VOvP7oe3TMz0O9++da911XVplHWM23IHwDTYNT5M0z2SN3lT0Tjs0ZH61U+r+syAGAoN9UNI3/OR/fM6EvXvWig3125/u51Iy5n6pA/AKbBqPNnmOyRusufiMYHADC/kjSr2a7LAAAsI6nZQ+MDANFKM5UXPgCAZJnZw+IGAAAAAKYeR3wAIFhvukF1XQamkFcd+E8Er17dUSXTZfZ73+sfLP4OI0tq9tD4AEC4xHnWAIBsidlD4wMAwUqlGfYWAwDGKDV7aHwAIFzidAMAQLbE7KHxAYBgJWkmMHwAALlSs4fGBwAALMirj+i6hDg1k3cOBDDNaHwAIFzidAMAQLbE7KHxAYBgJUWeYAoAyJWaPTQ+ABCOyTQAgHFLzB4aHwAIVqrIE0wBALlSs4fGBwCSlTSTlz0AgGSh2bOizSe3fZztq21/zfYO22fbXmv7ett3N38e32YNAIDB2L7C9m7btx/i/rfZvtX2bba/YPsV467xUMgfAMjVVv602vhI+pCkT1fVyyS9QtIOSZdKuqGqNkq6odkGAAyg1JtnPchtET4qadNh7v+mpNdV1csl/ZakLUt/B60hfwCgJcNkT5f501rjY/t5kn5c0kckqaqerqrHJV0oaWvzsK2S3tRWDQAw/ayZAW8LqarPS9pzmPu/UFWPNZtflLRhNO9pOOQPALRt8OzpMn/aPOJzqqRvSfpT21+xfbntoyWdWFUPNo95SNKJLdYAAFOtJM3WYLcRu0TS34z8WQdD/gBAi4bJni7zp83FDVZJOkPSO6vqJtsf0kHTCqqqbM/79m1vlrRZktboqBbLBIBsi9l7dgjrbG+fs72lqpY8Xc3269ULntcOWsiIkT8A0LIhskfqKH/abHx2SdpVVTc121erFzwP215fVQ/aXi9p93y/3Lz5LZJ0rNcGrhsBAO0rDRU+j1TVmcO8vu0flnS5pPOr6tFhnmuEyB8AaNGQ2SN1lD+tTXWrqock3W/7B5qh8yTdKWmbpIubsYslXdNWDQCA9th+kaS/kvSzVfX1ruvZj/wBgOk2aP60fR2fd0r6c9urJd0j6RfUa7ausn2JpPskvaXlGgBgqs3WUHvdDsn2xyWdo96UhF2S3ifpCEmqqj+W9F5JJ0j6sG1J2jfsHrwRIn8AoEVtZY/UXv602vhU1S2S5ivivDZfFwCWixFMNzj0c1ddtMD9vyjpF1t58SGRPwDQnjazR2ovf9o+4gMAaFHJmmn9kmwAAPyz1Oyh8QGAcG1ONwAAYD6J2UPjAwDB2p5uAADAwVKzJ+8YFQAAAAAsEUd8ACCaNVPswwIAjFNm9tD4AECwkjTLwXsAwBilZg+NDwCES5xnDQDIlpg9ND4AEKwqc7oBACBXavbQ+ABAuNnAvW4AgGyJ2ZPXqgEAAADAEnHEBwCC9a6lwD4sAMD4pGYPjQ8ARMucZw0ASJaZPTQ+ABAsdUlRAECu1Oyh8QGAcDOVd4IpACBbYvbQ+ABAsJIj51kDAHKlZk9exQAAAACwRBzxAYBws4EnmAIAsiVmD40PAARLXVIUAJArNXtofAAgWMmRJ5gCAHKlZg+NDwCES1xSFACQLTF7aHwAIFiVIi8iBwDIlZo9eRUDAAAAwBJxxAcAolmzyptnDQBIlpk9U9X47D3/R7suoRNHfWNP39jM17/RQSUAxq2UOd0AAJArNXumqvEBgOUocUlRAEC2xOyh8QGAYCVrNnBJUQBArtTsofEBgHCJe90AANkSsyevYgAAAABYoqk64vOdFx74dva8bm9HlbToiSP6hl72hx3UAWAilKTZwBNMAYzPqlNf3HUJQ9n3zfu6LgEHSc2eqWp8AGD5sWYClxQFACTLzB4aHwAIlrrXDQCQKzV7aHwAIFziXjcAQLbE7KHxAYBgVY7c6wYAyJWaPTQ+AAAAU2xm7TF9Y3f9uyM7qGRhL/z0yr6xY1jcACNC4wMA4WYC97oBALIlZg+NDwAEK0mzgfOsAQC5UrOHxgcAojlyrxsAIFlm9tD4AECw3pKieXvdAAC5UrNnqhqfox6ZOXD7E1P19hrVdQEAJsyM8va6AejWmmP39o0dc2T/2Pgd33UBWKTE7Gm1Ytv32r7N9i22tzdja21fb/vu5k8+4QAwgWxfYXu37dsPcb9t/6HtnbZvtX3GuGs8FPIHAHK1lT/jaNVeX1WnV9WZzfalkm6oqo2Sbmi2AQADKFmzNdhtET4qadNh7j9f0sbmtlnSHw39hkaL/AGAFgyTPV3mTxfHqC6UtLX5eaukN3VQAwBMjVmtGOi2kKr6vKQ9h3nIhZL+R/V8UdJxtteP6G21gfwBgBEZNHu6zJ+2T4IpSX9ruyT9SVVtkXRiVT3Y3P+QpBNH9WJHXvOlUT1VlJmFHwJgSlVJM92dYHqSpPvnbO9qxh6c/+FjNdb8AYDlpOPskQbMn7Ybn9dW1QO2XyDpettfm3tnVVUTSn1sb1bv0JXW6KiWywSAXEOsrLNu//kvjS1NgzANyB8AaNGQq7p1kj+tNj5V9UDz527bn5R0lqSHba+vqgebQ1K7D/G7WyRtkaRjvZalzABgHr151gPPWn5kzvkvg3hA0slztjc0Y50jfwCgPUNmj9RR/rR2jo/to20/d//Pkn5S0u2Stkm6uHnYxZKuaasGAECrtkn6uWZ1nVdL+qc5U8k6Q/4AwNQbKH/aPOJzoqRP2t7/Oh+rqk/bvlnSVbYvkXSfpLe0WAMATL0ZtTPP2vbHJZ2j3pSEXZLeJ+kISaqqP5Z0raSflrRT0ncl/UIrhSwd+QMALWsre6T28qe1xqeq7pH0innGH5V0XluvCwDLSZtXz66qixa4vyS9vZUXHwL5AwDtajN7pPbyp+3FDQAArRp6njUAAEuUmT00PgAQbrbF6QYAAMwnMXtofAAg2ARcSwEAsMykZk/eMSoAAAAAWCKO+ABAuMR51gCAbInZQ+MDAMF6F5HLm24AAMiVmj00PgAQLvEEUwBAtsTsofEBgGBtX0sBAICDpWYPjQ8AhEucZw0AyJaYPXkVAwAAAMASccQHAJJV5gmmAIBgodlD4wMAwUqZJ5gCAHKlZg+NDwCES9zrBgDIlpg9ND4AECx1ZR0AQK7U7KHxAYBwieGDyVezdcD2zBPf6agSAJMoMXtY1Q0AAADA1OOIDwAEK2WurAMAyJWaPTQ+ABAucWUdAEC2xOyh8QGAZJU5zxoBZme6rgDApArNHhofAAiWurIOACBXavbQ+ABAuMTwAQBkS8weVnUDAAAAMPU44gMAwVJX1gEA5ErNHhofAAhXgeEDAMiWmD00PgAQLnFJUQBAtsTsofEBgGAVuqQoACBXavbQ+ABAuMTpBgCAbInZw6puAAAAAKYeR3wAIFrmyjoA2rHv3B/pG9vzg8/pG/u3P3hd39gLj3i8lZr2+/A3X9c39vQ+/imaKTN7+LQBQLjE6QYAgGyJ2UPjAwDBSpknmAIAcqVmD40PACSr3uo6AACMTWj20PgAQLjEaykAALIlZg+NjyQfsbp/bGW7C97NPvVUq88PAACWn1rV/4/Rb5862zf2M8d+tW9s/apjRlbHl/Y+M7LnAkaFxgcAgpUyTzAFAORKzR4aHwCIlrmkKAAgWWb20PgAQLjEE0wBANkSs4fGBwDCJU43AABkS8weGp9DuPfSM/rGZtYc2Nq++af+flHP9a2nn9s3tn3rjw1WWAde8OEvdF0CgEOoygwfAO1Y+d19fWPP+/oRfWOX7Tm7b2zD6j2t1ITpk5o97S5dJsn2Sttfsf2pZvtU2zfZ3mn7L2z3L6kGAJgItjfZvqv5zr50nvtfZPvG5nv+Vts/3UWd8yF/ACBTW9nTeuMj6V2SdszZ/oCkD1bVSyQ9JumSMdQAAFNrtjzQbSG2V0q6TNL5kk6TdJHt0w562G9KuqqqXinprZI+POK3NwzyBwBaMmj2LJQ/bWZPq42P7Q2S/pWky5ttSzpX0tXNQ7ZKelObNQDAtKsa7LYIZ0naWVX3VNXTkq6UdOHBLy/p2Obn50n6f6N6X8MgfwCgXYNmzyLyp7Xsafscnz+Q9GuS9p/kcoKkx6tq/wTUXZJOarkGAJhqLc6zPknS/XO2d0l61UGP+U+S/tb2OyUdLekNbRWzROQPALQoMXtaa3xsv1HS7qr6su1zBvj9zZI2S9IaHTXi6gZTL/reAdt/uaN/AYT5rL6tv/4NLBgAYARKHiZ81tnePmd7S1VtWeJzXCTpo1X1e7bPlvRntn+oqvovFT8m05g/wGKt+Ltb+sae/71/2Tf2ibtP7xs75si9rdSE6TNk9kjD589A2dPmEZ/XSLqgOdlojXqHoz4k6Tjbq5q9bhskPTDfLzdvfoskHeu1gSuFA8B4DPEF+UhVnXmY+x+QdPKc7fm+sy+RtEmSquofbK+RtE7S7sHLGhr5AwAtG/LL8XD501r2tHaOT1W9p6o2VNUp6p109NmqepukGyW9uXnYxZKuaasGAMBQbpa0sVkNbbV63+XbDnrM/5VLine truncated
"text/plain": [
"<Figure size 864x360 with 4 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"vis.plot(generate_heat_example(batch(view_examples=2), spatial(x=64, y=64)));"
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"collapsed": false,
"id": "XURdC4v_S2eP"
},
"source": [ "source": [
"\n", "\n",
"## Differentiable physics and gradient descent\n", "## Differentiable physics and gradient descent\n",
@@ -174,14 +178,13 @@
"\n", "\n",
"For the implementation below, we will simply use `y = diffuse.fourier(x, 8., 1)` for the forward pass, and then similarly compute an $L^2$ loss for two $y$ fields to which `diffuse.fourier( , 8., 1)` is applied.\n", "For the implementation below, we will simply use `y = diffuse.fourier(x, 8., 1)` for the forward pass, and then similarly compute an $L^2$ loss for two $y$ fields to which `diffuse.fourier( , 8., 1)` is applied.\n",
"\n" "\n"
], ]
"metadata": {
"collapsed": false,
"id": "XURdC4v_S2eP"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "EUIM1Av5vlfR"
},
"source": [ "source": [
"## Stable SIP gradients\n", "## Stable SIP gradients\n",
"\n", "\n",
@@ -190,7 +193,9 @@
"The frequency formulation of the heat equation can be inverted analytically, yielding\n", "The frequency formulation of the heat equation can be inverted analytically, yielding\n",
"$\\hat x_k = \\hat y_k \\cdot e^{k^2 (t_* - t_0)}$.\n", "$\\hat x_k = \\hat y_k \\cdot e^{k^2 (t_* - t_0)}$.\n",
"This allows us to define the update \n", "This allows us to define the update \n",
"\n",
"$$\\Delta x_{\\text{PG}} = - \\eta \\cdot \\mathcal F^{-1}\\left( e^{k^2 (t_* - t_0)} \\mathcal F(y - y^*) \\right).$$\n", "$$\\Delta x_{\\text{PG}} = - \\eta \\cdot \\mathcal F^{-1}\\left( e^{k^2 (t_* - t_0)} \\mathcal F(y - y^*) \\right).$$\n",
"\n",
"Here, high frequencies are multiplied by exponentially large factors, resulting in numerical instabilities.\n", "Here, high frequencies are multiplied by exponentially large factors, resulting in numerical instabilities.\n",
"When applying this formula directly to the gradients, it can lead to large oscillations in $\\Delta x_{\\text{PG}}$.\n", "When applying this formula directly to the gradients, it can lead to large oscillations in $\\Delta x_{\\text{PG}}$.\n",
"\n", "\n",
@@ -209,14 +214,17 @@
"This leads to a much faster convergence and more precise solution than any generic optimization method.\n", "This leads to a much faster convergence and more precise solution than any generic optimization method.\n",
"The cell below implements this probabilistic approach, with `probability_signal` in `apply_damping()` containing the parts of the signal to be dampened.\n", "The cell below implements this probabilistic approach, with `probability_signal` in `apply_damping()` containing the parts of the signal to be dampened.\n",
"The `inv_diffuse()` functions employs it to compute a stabilized inverse diffusion process.\n" "The `inv_diffuse()` functions employs it to compute a stabilized inverse diffusion process.\n"
], ]
"metadata": {
"id": "EUIM1Av5vlfR"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 20, "execution_count": null,
"metadata": {
"id": "jJFlJlRwS2eP",
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"def apply_damping(kernel, inv_kernel, amp, f_uncertainty, log_kernel):\n", "def apply_damping(kernel, inv_kernel, amp, f_uncertainty, log_kernel):\n",
@@ -247,16 +255,14 @@
" uncertainty = math.sqrt(math.sum(((f_uncertainty * kernel) ** 2))) / grid.shape.get_size('x') # 1/N normalization in iFFT\n", " uncertainty = math.sqrt(math.sum(((f_uncertainty * kernel) ** 2))) / grid.shape.get_size('x') # 1/N normalization in iFFT\n",
" uncertainty = grid * 0 + uncertainty\n", " uncertainty = grid * 0 + uncertainty\n",
" return grid.with_values(data), uncertainty, abs(amp), raw_kernel, kernel, sig_prob" " return grid.with_values(data), uncertainty, abs(amp), raw_kernel, kernel, sig_prob"
], ]
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "jJFlJlRwS2eP"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"collapsed": false,
"id": "Prft7mS6S2eQ"
},
"source": [ "source": [
"## Neural network and loss function\n", "## Neural network and loss function\n",
"\n", "\n",
@@ -264,15 +270,17 @@
"We train with a batch size of 128 and a constant learning rate of $\\eta = 10^{-3}$, using 64 bit precision for physics but 32 bits for the network.\n", "We train with a batch size of 128 and a constant learning rate of $\\eta = 10^{-3}$, using 64 bit precision for physics but 32 bits for the network.\n",
"The network updates are computed with TensorFlow's or PyTorch's automatic differentiation.\n", "The network updates are computed with TensorFlow's or PyTorch's automatic differentiation.\n",
"\n" "\n"
], ]
"metadata": {
"collapsed": false,
"id": "Prft7mS6S2eQ"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 21, "execution_count": null,
"metadata": {
"id": "edLI4HnjS2eR",
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"math.set_global_precision(64)\n", "math.set_global_precision(64)\n",
@@ -282,16 +290,14 @@
"math.seed(0)\n", "math.seed(0)\n",
"net = u_net(1, 1)\n", "net = u_net(1, 1)\n",
"optimizer = adam(net, 0.001)" "optimizer = adam(net, 0.001)"
], ]
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "edLI4HnjS2eR"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"collapsed": false,
"id": "lkkXZc_kS2eR"
},
"source": [ "source": [
"Now we'll define loss function for phiflow.\n", "Now we'll define loss function for phiflow.\n",
"The gradients for the network weights are always computed as d(loss_function)/d$\\theta$.\n", "The gradients for the network weights are always computed as d(loss_function)/d$\\theta$.\n",
@@ -301,15 +307,17 @@
"The Adam / GD version for `sip=False` simply computes the L2 difference of the predicted, diffused $y$ field to the target, and backprops through the operations for the gradient.\n", "The Adam / GD version for `sip=False` simply computes the L2 difference of the predicted, diffused $y$ field to the target, and backprops through the operations for the gradient.\n",
"\n", "\n",
"We also compute the reference $y = \\mathcal P(x^*)$ in the loss function on-the-fly.\n" "We also compute the reference $y = \\mathcal P(x^*)$ in the loss function on-the-fly.\n"
], ]
"metadata": {
"collapsed": false,
"id": "lkkXZc_kS2eR"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 22, "execution_count": null,
"metadata": {
"id": "aUKHZMwkS2eR",
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"# @math.jit_compile\n", "# @math.jit_compile\n",
@@ -329,57 +337,39 @@
" y = diffuse.fourier(prediction, 8., 1)\n", " y = diffuse.fourier(prediction, 8., 1)\n",
" loss = y_l2 = field.l2_loss(y - y_target) # for Adam we backprop through the loss and y\n", " loss = y_l2 = field.l2_loss(y - y_target) # for Adam we backprop through the loss and y\n",
" return loss, x, y, field.l2_loss(x_gt - x), y_l2\n" " return loss, x, y, field.l2_loss(x_gt - x), y_l2\n"
], ]
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "aUKHZMwkS2eR"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"collapsed": false,
"id": "0L142o1OS2eS"
},
"source": [ "source": [
"## Training\n", "## Training\n",
"\n", "\n",
"In the training loop, we generate data on-the-fly via `generate_heat_example()` and use phiflow's `update_weights()` function to call the correct functions of the chosen backend. Note that we're only printing the loss of the first 5 steps below for clarity, the whole history is collected in the `loss_` lists. The following cell runs the SIP version of the training:\n" "In the training loop, we generate data on-the-fly via `generate_heat_example()` and use phiflow's `update_weights()` function to call the correct functions of the chosen backend. Note that we're only printing the loss of the first 5 steps below for clarity, the whole history is collected in the `loss_` lists. The following cell runs the SIP version of the training:\n"
], ]
"metadata": {
"collapsed": false,
"id": "0L142o1OS2eS"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"source": [ "execution_count": null,
"loss_sip_x=[]; loss_sip_y=[] \n",
"for training_step in range(STEPS):\n",
" data = generate_heat_example(spatial(x=64, y=64), BATCH)\n",
" loss_value, x_sip, y_sip, x_l2, y_l2 = update_weights(net, optimizer, loss_function, net, data, sip=True)\n",
" loss_sip_x.append(float(x_l2.mean)) \n",
" loss_sip_y.append(float(y_l2.mean))\n",
" if(training_step<5): print(\"SIP L2 loss x ; y: \"+format(float(x_l2.mean))+\" ; \"+format(float(y_l2.mean)) )"
],
"metadata": { "metadata": {
"colab": { "colab": {
"base_uri": "https://localhost:8080/" "base_uri": "https://localhost:8080/"
}, },
"id": "PK47PUNKaeA0", "id": "PK47PUNKaeA0",
"outputId": "f275a0bb-6fab-412a-fcc7-e2a786b8fa8b" "outputId": "4d938ed4-484b-46df-d0ca-d2268cb97dd0"
}, },
"execution_count": 23,
"outputs": [ "outputs": [
{ {
"output_type": "stream",
"name": "stderr", "name": "stderr",
"text": [ "output_type": "stream",
"/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:3635: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.\n", "text": []
" \"See the documentation of nn.Upsample for details.\".format(mode)\n"
]
}, },
{ {
"output_type": "stream",
"name": "stdout", "name": "stdout",
"output_type": "stream",
"text": [ "text": [
"SIP L2 loss x ; y: 187.2586057892786 ; 59.48433060883144\n", "SIP L2 loss x ; y: 187.2586057892786 ; 59.48433060883144\n",
"SIP L2 loss x ; y: 70.21347147390776 ; 13.783476203544797\n", "SIP L2 loss x ; y: 70.21347147390776 ; 13.783476203544797\n",
@@ -388,32 +378,48 @@
"SIP L2 loss x ; y: 34.611490797378366 ; 3.359133206049436\n" "SIP L2 loss x ; y: 34.611490797378366 ; 3.359133206049436\n"
] ]
} }
],
"source": [
"loss_sip_x=[]; loss_sip_y=[] \n",
"for training_step in range(STEPS):\n",
" data = generate_heat_example(spatial(x=64, y=64), BATCH)\n",
" loss_value, x_sip, y_sip, x_l2, y_l2 = update_weights(net, optimizer, loss_function, net, data, sip=True)\n",
" loss_sip_x.append(float(x_l2.mean)) \n",
" loss_sip_y.append(float(y_l2.mean))\n",
" if(training_step<5): print(\"SIP L2 loss x ; y: \"+format(float(x_l2.mean))+\" ; \"+format(float(y_l2.mean)) )"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"source": [
"And now we can repeat the training with the DP version using Adam, deactivating the SIP update via `sip=False`:"
],
"metadata": { "metadata": {
"id": "32HFjc2y7noA" "id": "32HFjc2y7noA"
}
}, },
{ "source": [
"cell_type": "code", "And now we can repeat the training with the DP version using Adam, deactivating the SIP update via `sip=False`:"
"execution_count": 24,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:3635: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.\n",
" \"See the documentation of nn.Upsample for details.\".format(mode)\n"
] ]
}, },
{ {
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0ENSdgZiS2eS",
"outputId": "9262848f-2838-4fc5-f36c-0a1d88a0f7af",
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": []
},
{
"name": "stdout", "name": "stdout",
"output_type": "stream",
"text": [ "text": [
"GD L2 loss x ; y: 187.2586057892786 ; 59.48433060883144\n", "GD L2 loss x ; y: 187.2586057892786 ; 59.48433060883144\n",
"GD L2 loss x ; y: 104.37856249315794 ; 20.323700695817763\n", "GD L2 loss x ; y: 104.37856249315794 ; 20.323700695817763\n",
@@ -435,33 +441,46 @@
" loss_gd_x.append(float(x_l2.mean)) \n", " loss_gd_x.append(float(x_l2.mean)) \n",
" loss_gd_y.append(float(y_l2.mean))\n", " loss_gd_y.append(float(y_l2.mean))\n",
" if(training_step<5): print(\"GD L2 loss x ; y: \"+format(float(x_l2.mean))+\" ; \"+format(float(y_l2.mean)) )" " if(training_step<5): print(\"GD L2 loss x ; y: \"+format(float(x_l2.mean))+\" ; \"+format(float(y_l2.mean)) )"
], ]
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "0ENSdgZiS2eS",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "272f48e4-fadd-44d6-c8b3-3704c8135a9a"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "eV1j8lYuwUqB"
},
"source": [ "source": [
"## Evaluation\n", "## Evaluation\n",
"\n", "\n",
"Now we can evaluate how the two variants behave in direct comparison. Note that due to the on-the-fly generation of randomized data, all samples are previously unseen, and hence we'll directly use the training curves here to draw conclusions about the performance of the two approaches.\n", "Now we can evaluate how the two variants behave in direct comparison. Note that due to the on-the-fly generation of randomized data, all samples are previously unseen, and hence we'll directly use the training curves here to draw conclusions about the performance of the two approaches.\n",
"\n", "\n",
"The following graph shows the $L^2$ error over training in terms of the reconstructed $x$ input, the main goal of our training." "The following graph shows the $L^2$ error over training in terms of the reconstructed $x$ input, the main goal of our training."
], ]
"metadata": {
"id": "eV1j8lYuwUqB"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"id": "uUkHS8Xljn7O",
"outputId": "b20fa929-c474-4b4c-be42-00f382c2fee8"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEGCAYAAABlxeIAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZdoG8PtJAoQSQgkg0gIiCggSmiLFgMoiLitYwLKKu7iuwqpgYdX1w7Ki6OoKlrVhQVdQV0UR6U1AkA5RihTpnQCBEAIheb4/npnMpEwySaZm7t91nWtmzpzyHpic57xdVBVERERRwU4AERGFBgYEIiICwIBAREQODAhERASAAYGIiBxigp2AskhISNDExMRgJ4OIKKysWrXqiKrWyb8+rANCYmIiVq5cGexkEBGFFRHZWdh6FhkREREABgQiInJgQCAiIgBhWocgIv0A9GvevHmwk0JE5UxWVhb27NmDzMzMYCelzGJjY9GwYUNUqFDBq+0lnMcy6tixo7JSmYh8afv27YiLi0Pt2rUhIsFOTqmpKlJTU3Hy5Ek0bdo0z3ciskpVO+bfh0VGRERuMjMzwz4YAICIoHbt2iXK6TAgEBHlE+7BwKmk1xFxAeHgz4ewuk5vrKjbN9hJISIKKREXEKomVEb7I7PR6vAPyM4OdmqIiAp38OBB3HbbbWjWrBk6dOiALl26YPLkyViwYAHi4+ORlJSEiy66CD169MDUqVN9cs6wbGVUFtXOq4bTUhlVNQM7NqYj8ZJqwU4SEVEeqor+/ftj8ODBmDhxIgBg586dmDJlCmrWrInu3bvnBoG1a9eif//+qFy5Mq666qoynTficggQQVrFugCA3asOBTkxREQFzZs3DxUrVsS9996bu65Jkya4//77C2zbrl07jBo1Cm+88UaZzxt5AQHA6eoWEA6kMCAQkWci/lmKs379erRv397rdLZv3x6bNm0qw5WaiAwIObUtIBzfzIBARKFv2LBhuPTSS9GpU6dCv/dVf7KIDAgxDeoBADJ2HAxySogolKn6ZylO69atsXr16tzPb775JubOnYvDhw8Xuv2aNWvQsmXLMl9vRAaEKomWQzi3jzkEIgo9vXr1QmZmJt56663cdRkZGYVum5KSgn/+858YNmxYmc8bca2MAKBGCwsIFY4dQlYW4OUwH0REASEi+OabbzBixAi89NJLqFOnDqpWrYoXX3wRALBo0SIkJSUhIyMDdevWxWuvvVbmFkZAhAaECg0sINTRQ9i+HWjRIsgJIiLKp379+vjss88K/S4tLc0v54zIIiPUszqEejiIzZuDnBYiohARmQGhruUQ6uIQtmwJclqIiEIEAwIDAhERgEgNCAkJUBEk4Ai2beaARkREQKQGhJgYZNeojSgojmw6EuzUEBGFhMgMCACiz7Nio6y9h1AOZsojIiqziA0IUs/R9BSHsG1bkBNDRJTP6NGj0bp1a7Rt2xbt2rXDsmXLkJycDOe0wYmJiWjTpg3atm2L3r1748CBA2U+Z8QGBPeKZTY9JaJQsnTpUkydOhWrV69GSkoK5syZg0aNGhXYbv78+UhJSUHHjh3x/PPPl/m8kRsQ3PoisKUREYWS/fv3IyEhAZUqVQIAJCQk4Pzzz/e4fY8ePbB169YynzdyAwKbnhJRcYI0/nXv3r2xe/dutGjRAkOHDsUPP/xQ5PZTp05FmzZtyny5DAgsMiKiEFOtWjWsWrUK7777LurUqYNBgwbho48+KrBdz5490a5dO5w4cQKPP/54mc8bkWMZAWAOgYiK56N5BkojOjoaycnJSE5ORps2bTBhwoQC28yfPx8JCQk+O2fk5hAcdQjnyUHs3w+kpwc5PUREDr/++iu2uD2prl27Fk2aNPH7eSM3IDhyCOfH2JwIzCUQUahIT0/H4MGD0apVK7Rt2xYbNmzA008/7ffzRnyRUUKOKyAkJQUzQUREpkOHDliyZEmB9QsWLMh9v2PHDp+fN3JzCNWqAbGxiM3OQBWcYg6BiCJe5AYEEfZFICJyE7kBAWDTUyIqlAaxdZEvlfQ6GBDApqdE5BIbG4vU1NSwDwqqitTUVMTGxnq9T+RWKgO5AaFhxUM4cgQ4dgyoWTPIaSKioGrYsCH27NmDw4cPBzspZRYbG4uGDRt6vT0DAoBWtQ4CB6ylUefOQU4TEQVVhQoV0LRp02AnIygiu8jIUancrBr7IhARRXZAcOQQGlRkQCAiYkAAUEctILClERFFMgYEAPGZBwEwh0BEkS2yA4KjDqHySVeRUZi3NCMiKrXIDgiOYWMl9QhqVs9GWhpQDlqaERGVSmQHhJgYoHZtiCo6Nk0FwGIjIopckR0QgNx6hPYNWI9ARJGNAcFRj9CyNpueElFkY0Bw5BAuiGPTUyKKbAwIjoDQOJY5BCKKbAwIzhFP1VWHwKanRBSJGBAcdQixJw4hIQHIyAD27AlymoiIgiBkAoKI9BeR90TkcxHpHbATO3IIOHQI7drZ2+XLA3Z2IqKQ4deAICIfiMghEfkl3/o+IvKriGwVkccAQFW/UdW/ALgXwCB/pisPt4DQrZu9Xbw4YGcnIgoZ/s4hfASgj/sKEYkG8CaAawG0AnCriLRy2+RJx/eB4QwIBw/mBoQffwzY2YmIQoZfA4KqLgRwNN/qzgC2qupvqnoWwGcArhfzIoDpqrra0zFF5B4RWSkiK30yo5GjDgGHDuGyy4DoaGD1auDUqbIfmogonASjDqEBgN1un/c41t0P4GoAN4nIvZ52VtV3VbWjqnasU6dO2VNTrRoQGwtkZKCanEJSEpCdDSxbVvZDExGFk5CpVFbV11S1g6req6pvB+zEIqxHICJCcALCXgCN3D43dKwLHrd6hK5d7S0DAhFFmmAEhBUALhSRpiJSEcAtAKYEIR0ubvUIzoCwdClw7lzwkkREFGj+bnY6CcBSABeJyB4RGaKq5wD8DcBMABsBfKGq6/2ZjmK5FRnVrw9ccAGQng6kpAQ1VUREARXjz4Or6q0e1k8DMM2f5y4Rt4AAAN26Adu2WbFR+/ZBTBcRUQCFTKVyULnVIQBgxTIRRaSwDAgi0k9E3k1LS/PNAd3qEIC8AYED3RFRpAjLgKCq36nqPfHx8b45YL4io4suAmrXBvbvB7Zv980piIhCXVgGBJ/LFxBEkNvaiMNYEFGkYEAACtQhAKxHIKLIw4AAAM4hMI4csXErwIBARJGHAQEAYmKs0kAVSE0FYM1NY2OBDRtyVxERlWsMCE756hEqVQI6d7ZVS5YEKU1ERAHEgODEegQiinBhGRB83g8BKNAXAWBAIKLIEpYBwef9EIACRUYA0KWLNUFdsQI4fdp3pyIiCkVhGRD8opCAUKMGcMklQFYWsHJlkNJFRBQgDAhOhdQhACw2IqLIwYDgVEgdAsCAQESRgwHBqZAiI8AVEJYsAXJyApwmIqIAYkBw8hAQGjcGGjUCjh+3TmpEROUVA4KThzoEAOjRw16/+iqA6SEiCjAGBKe4OBurIiMDOHUqz1dDhtjrW28BZ84EIW1ERAHAgOAk4sol7N2b56vkZKBtW8s8TJoU+KQREQVCWAYEv/RUBlwTKC9YkO98wIgR9n7sWM6iRkTlU1gGBL/0VAaAPn3sdcaMAl/dcotlINatKxAviIjKhbAMCH7jDAhz5lj3ZDexscDQofZ+7NgAp4uIKAAYENw1aQK0bAmcPAksXVrg6/vuAypWBL77Dti6NQjpIyLyIwaE/Jy5hOnTC3xVty5w++1WhzBuXIDTRUTkZwwI+RVRjwC4Kpc//NA6qxERlRcMCPn16AFUrgysXQvs31/g6zZtgKuusq4K48cHIX1ERH7CgJBfbKx1PACAWbMK3WT4cHt9/XXg3LnAJIuIyN8YEApz7bX26qHYqG9foEULYNcuYPLkAKaLiMiPGBAK46xHmDULyM4u8HVUFPDgg/b+1VcDmC4iIj9iQChM8+ZAs2bA0aM2f2Yh7rzTZlRbuhRYtizA6SMi8oOwDAh+G7rCdYJiWxtVqwbcc4+9HzPGP8kgIgqksAwIfhu6wl0x9QiAVS7HxgLffAOsWeO/pBARBUJYBoSASE62bsnLlwNHjhS6Sf36ruEsnn46YCkjIvILBgRPqlUDune3bsmzZ3vcbORIoEoVYMoUYOXKAKaPiMjHig0IIvKgiFQX876IrBaR3oFIXNAVU48AAPXqAX/7m71/6qkApImIyE+8ySH8WVVPAOgNoCaAOwBERjWqsx5h5kwgJ8fjZo8+ClStCkybBvz0U4DSRkTkY94EBHG89gXwiaqud1tXvrVqBTRsaFOlrVvncbOEBOCBB+w96xKIKFx5ExBWicgsWECYKSJxADw/Lpcn7s1PCxn91N3DD9u0zDNnAj/+GIC0ERH5mDcBYQiAxwB0UtUMABUA/MmvqQolXtQjAEDt2q4xjliXQEThyJuA0AXAr6p6XET+COBJAH7qERaCrr4aiI4GliwBiukIN2IEEB8PzJ0LLFwYoPQREfmINwHhLQAZInIpgIcBbAPwsV9TFUri44ErrrAxjYpofgoANWu65ktgLoGIwo03AeGcqiqA6wG8oapvAojzb7JCzHXX2euUKcVuOny4jXG0YAEwf75/k0VE5EveBISTIvI4rLnp9yISBatHiBz9+9vr1KlAVlaRm8bHA488Yu+HDfPYyZmIKOR4ExAGATgD649wAEBDAP/ya6pCzUUX2XLsGLB4cbGbP/CAtVjduBHo3ZtTbRJReCg2IDiCwKcA4kXk9wAyVTWodQh+H+20MM5cwjffFLtpXBwwZ46Nor1mjfVvO3nSz+kjIiojb4auGAhgOYCbAQwEsExEbvJ3wooSkNFO87v+env99lsb36gY9etba6PGja33cr9+QEaGn9NIRFQG3hQZ/QPWB2Gwqt4JoDOA//NvskLQZZfZwEU7dxbZa9ld48bAvHnA+ecDP/wLine truncated
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [ "source": [
"import pylab as plt\n", "import pylab as plt\n",
"fig = plt.figure().gca()\n", "fig = plt.figure().gca()\n",
@@ -469,44 +488,44 @@
"fig.plot(pltx, loss_gd_x , lw=2, color='blue', label=\"GD\") \n", "fig.plot(pltx, loss_gd_x , lw=2, color='blue', label=\"GD\") \n",
"fig.plot(pltx, loss_sip_x , lw=2, color='red', label=\"SIP\")\n", "fig.plot(pltx, loss_sip_x , lw=2, color='red', label=\"SIP\")\n",
"plt.xlabel('iterations'); plt.ylabel('x loss'); plt.legend(); plt.yscale(\"log\")\n" "plt.xlabel('iterations'); plt.ylabel('x loss'); plt.legend(); plt.yscale(\"log\")\n"
],
"metadata": {
"id": "uUkHS8Xljn7O",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"outputId": "aee621e7-d572-44fb-8100-43988e999ae1"
},
"execution_count": 25,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEGCAYAAABlxeIAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZdoG8PtJAoQSQgkg0gIiCggSmiLFgMoiLitYwLKKu7iuwqpgYdX1w7Ki6OoKlrVhQVdQV0UR6U1AkA5RihTpnQCBEAIheb4/npnMpEwySaZm7t91nWtmzpzyHpic57xdVBVERERRwU4AERGFBgYEIiICwIBAREQODAhERASAAYGIiBxigp2AskhISNDExMRgJ4OIKKysWrXqiKrWyb8+rANCYmIiVq5cGexkEBGFFRHZWdh6FhkREREABgQiInJgQCAiIgBhWocgIv0A9GvevHmwk0JE5UxWVhb27NmDzMzMYCelzGJjY9GwYUNUqFDBq+0lnMcy6tixo7JSmYh8afv27YiLi0Pt2rUhIsFOTqmpKlJTU3Hy5Ek0bdo0z3ciskpVO+bfh0VGRERuMjMzwz4YAICIoHbt2iXK6TAgEBHlE+7BwKmk1xFxAeHgz4ewuk5vrKjbN9hJISIKKREXEKomVEb7I7PR6vAPyM4OdmqIiAp38OBB3HbbbWjWrBk6dOiALl26YPLkyViwYAHi4+ORlJSEiy66CD169MDUqVN9cs6wbGVUFtXOq4bTUhlVNQM7NqYj8ZJqwU4SEVEeqor+/ftj8ODBmDhxIgBg586dmDJlCmrWrInu3bvnBoG1a9eif//+qFy5Mq666qoynTficggQQVrFugCA3asOBTkxREQFzZs3DxUrVsS9996bu65Jkya4//77C2zbrl07jBo1Cm+88UaZzxt5AQHA6eoWEA6kMCAQkWci/lmKs379erRv397rdLZv3x6bNm0qw5WaiAwIObUtIBzfzIBARKFv2LBhuPTSS9GpU6dCv/dVf7KIDAgxDeoBADJ2HAxySogolKn6ZylO69atsXr16tzPb775JubOnYvDhw8Xuv2aNWvQsmXLMl9vRAaEKomWQzi3jzkEIgo9vXr1QmZmJt56663cdRkZGYVum5KSgn/+858YNmxYmc8bca2MAKBGCwsIFY4dQlYW4OUwH0REASEi+OabbzBixAi89NJLqFOnDqpWrYoXX3wRALBo0SIkJSUhIyMDdevWxWuvvVbmFkZAhAaECg0sINTRQ9i+HWjRIsgJIiLKp379+vjss88K/S4tLc0v54zIIiPUszqEejiIzZuDnBYiohARmQGhruUQ6uIQtmwJclqIiEIEAwIDAhERgEgNCAkJUBEk4Ai2beaARkREQKQGhJgYZNeojSgojmw6EuzUEBGFhMgMCACiz7Nio6y9h1AOZsojIiqziA0IUs/R9BSHsG1bkBNDRJTP6NGj0bp1a7Rt2xbt2rXDsmXLkJycDOe0wYmJiWjTpg3atm2L3r1748CBA2U+Z8QGBPeKZTY9JaJQsnTpUkydOhWrV69GSkoK5syZg0aNGhXYbv78+UhJSUHHjh3x/PPPl/m8kRsQ3PoisKUREYWS/fv3IyEhAZUqVQIAJCQk4Pzzz/e4fY8ePbB169YynzdyAwKbnhJRcYI0/nXv3r2xe/dutGjRAkOHDsUPP/xQ5PZTp05FmzZtyny5DAgsMiKiEFOtWjWsWrUK7777LurUqYNBgwbho48+KrBdz5490a5dO5w4cQKPP/54mc8bkWMZAWAOgYiK56N5BkojOjoaycnJSE5ORps2bTBhwoQC28yfPx8JCQk+O2fk5hAcdQjnyUHs3w+kpwc5PUREDr/++iu2uD2prl27Fk2aNPH7eSM3IDhyCOfH2JwIzCUQUahIT0/H4MGD0apVK7Rt2xYbNmzA008/7ffzRnyRUUKOKyAkJQUzQUREpkOHDliyZEmB9QsWLMh9v2PHDp+fN3JzCNWqAbGxiM3OQBWcYg6BiCJe5AYEEfZFICJyE7kBAWDTUyIqlAaxdZEvlfQ6GBDApqdE5BIbG4vU1NSwDwqqitTUVMTGxnq9T+RWKgO5AaFhxUM4cgQ4dgyoWTPIaSKioGrYsCH27NmDw4cPBzspZRYbG4uGDRt6vT0DAoBWtQ4CB6ylUefOQU4TEQVVhQoV0LRp02AnIygiu8jIUancrBr7IhARRXZAcOQQGlRkQCAiYkAAUEctILClERFFMgYEAPGZBwEwh0BEkS2yA4KjDqHySVeRUZi3NCMiKrXIDgiOYWMl9QhqVs9GWhpQDlqaERGVSmQHhJgYoHZtiCo6Nk0FwGIjIopckR0QgNx6hPYNWI9ARJGNAcFRj9CyNpueElFkY0Bw5BAuiGPTUyKKbAwIjoDQOJY5BCKKbAwIzhFP1VWHwKanRBSJGBAcdQixJw4hIQHIyAD27AlymoiIgiBkAoKI9BeR90TkcxHpHbATO3IIOHQI7drZ2+XLA3Z2IqKQ4deAICIfiMghEfkl3/o+IvKriGwVkccAQFW/UdW/ALgXwCB/pisPt4DQrZu9Xbw4YGcnIgoZ/s4hfASgj/sKEYkG8CaAawG0AnCriLRy2+RJx/eB4QwIBw/mBoQffwzY2YmIQoZfA4KqLgRwNN/qzgC2qupvqnoWwGcArhfzIoDpqrra0zFF5B4RWSkiK30yo5GjDgGHDuGyy4DoaGD1auDUqbIfmogonASjDqEBgN1un/c41t0P4GoAN4nIvZ52VtV3VbWjqnasU6dO2VNTrRoQGwtkZKCanEJSEpCdDSxbVvZDExGFk5CpVFbV11S1g6req6pvB+zEIqxHICJCcALCXgCN3D43dKwLHrd6hK5d7S0DAhFFmmAEhBUALhSRpiJSEcAtAKYEIR0ubvUIzoCwdClw7lzwkkREFGj+bnY6CcBSABeJyB4RGaKq5wD8DcBMABsBfKGq6/2ZjmK5FRnVrw9ccAGQng6kpAQ1VUREARXjz4Or6q0e1k8DMM2f5y4Rt4AAAN26Adu2WbFR+/ZBTBcRUQCFTKVyULnVIQBgxTIRRaSwDAgi0k9E3k1LS/PNAd3qEIC8AYED3RFRpAjLgKCq36nqPfHx8b45YL4io4suAmrXBvbvB7Zv980piIhCXVgGBJ/LFxBEkNvaiMNYEFGkYEAACtQhAKxHIKLIw4AAAM4hMI4csXErwIBARJGHAQEAYmKs0kAVSE0FYM1NY2OBDRtyVxERlWsMCE756hEqVQI6d7ZVS5YEKU1ERAHEgODEegQiinBhGRB83g8BKNAXAWBAIKLIEpYBwef9EIACRUYA0KWLNUFdsQI4fdp3pyIiCkVhGRD8opCAUKMGcMklQFYWsHJlkNJFRBQgDAhOhdQhACw2IqLIwYDgVEgdAsCAQESRgwHBqZAiI8AVEJYsAXJyApwmIqIAYkBw8hAQGjcGGjUCjh+3TmpEROUVA4KThzoEAOjRw16/+iqA6SEiCjAGBKe4OBurIiMDOHUqz1dDhtjrW28BZ84EIW1ERAHAgOAk4sol7N2b56vkZKBtW8s8TJoU+KQREQVCWAYEv/RUBlwTKC9YkO98wIgR9n7sWM6iRkTlU1gGBL/0VAaAPn3sdcaMAl/dcotlINatKxAviIjKhbAMCH7jDAhz5lj3ZDexscDQofZ+7NgAp4uIKAAYENw1aQK0bAmcPAksXVrg6/vuAypWBL77Dti6NQjpIyLyIwaE/Jy5hOnTC3xVty5w++1WhzBuXIDTRUTkZwwI+RVRjwC4Kpc//NA6qxERlRcMCPn16AFUrgysXQvs31/g6zZtgKuusq4K48cHIX1ERH7CgJBfbKx1PACAWbMK3WT4cHt9/XXg3LnAJIuIyN8YEApz7bX26qHYqG9foEULYNcuYPLkAKaLiMiPGBAK46xHmDULyM4u8HVUFPDgg/b+1VcDmC4iIj9iQChM8+ZAs2bA0aM2f2Yh7rzTZlRbuhRYtizA6SMi8oOwDAh+G7rCdYJiWxtVqwbcc4+9HzPGP8kgIgqksAwIfhu6wl0x9QiAVS7HxgLffAOsWeO/pBARBUJYBoSASE62bsnLlwNHjhS6Sf36ruEsnn46YCkjIvILBgRPqlUDune3bsmzZ3vcbORIoEoVYMoUYOXKAKaPiMjHig0IIvKgiFQX876IrBaR3oFIXNAVU48AAPXqAX/7m71/6qkApImIyE+8ySH8WVVPAOgNoCaAOwBERjWqsx5h5kwgJ8fjZo8+ClStCkybBvz0U4DSRkTkY94EBHG89gXwiaqud1tXvrVqBTRsaFOlrVvncbOEBOCBB+w96xKIKFx5ExBWicgsWECYKSJxADw/Lpcn7s1PCxn91N3DD9u0zDNnAj/+GIC0ERH5mDcBYQiAxwB0UtUMABUA/MmvqQolXtQjAEDt2q4xjliXQEThyJuA0AXAr6p6XET+COBJAH7qERaCrr4aiI4GliwBiukIN2IEEB8PzJ0LLFwYoPQREfmINwHhLQAZInIpgIcBbAPwsV9TFUri44ErrrAxjYpofgoANWu65ktgLoGIwo03AeGcqiqA6wG8oapvAojzb7JCzHXX2euUKcVuOny4jXG0YAEwf75/k0VE5EveBISTIvI4rLnp9yISBatHiBz9+9vr1KlAVlaRm8bHA488Yu+HDfPYyZmIKOR4ExAGATgD649wAEBDAP/ya6pCzUUX2XLsGLB4cbGbP/CAtVjduBHo3ZtTbRJReCg2IDiCwKcA4kXk9wAyVTWodQh+H+20MM5cwjffFLtpXBwwZ46Nor1mjfVvO3nSz+kjIiojb4auGAhgOYCbAQwEsExEbvJ3wooSkNFO87v+env99lsb36gY9etba6PGja33cr9+QEaGn9NIRFQG3hQZ/QPWB2Gwqt4JoDOA//NvskLQZZfZwEU7dxbZa9ld48bAvHnA+Line truncated
},
"metadata": {
"needs_background": "light"
}
}
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "-ZIRzk7ZwZ2x"
},
"source": [ "source": [
"The log-scale for the loss in $x$ nicely highlights that the SIP version does inherently better, and shows a significantly improved convergence for the NN training. This is purely caused by the better signal for the physics via the proxy loss. As shown in `loss_function()`, both variants use the same backprop-based update to change the weights of the NN. The improvements purely stem from the higher-order step in $y$ space computed by the inverse simulator.\n", "The log-scale for the loss in $x$ nicely highlights that the SIP version does inherently better, and shows a significantly improved convergence for the NN training. This is purely caused by the better signal for the physics via the proxy loss. As shown in `loss_function()`, both variants use the same backprop-based update to change the weights of the NN. The improvements purely stem from the higher-order step in $y$ space computed by the inverse simulator.\n",
"\n", "\n",
"Just out of curiosity, we can also compare how the two versions compare in terms of difference in the output space $y$." "Just out of curiosity, we can also compare how the two versions compare in terms of difference in the output space $y$."
], ]
"metadata": {
"id": "-ZIRzk7ZwZ2x"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"id": "KcJemrzNu6dm",
"outputId": "84bf4c24-9d8b-4f4a-a0d4-fbb6dfa569d8"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3debyWc/7H8den076fNtp0aloUJXVCkxKRSGMXWWKMBjEzmCHMDMOYYRiMYYzsP4wYM4xCSdok0W5JC0ml5UhpU53q+/vjc5+lOqfOdp/r3Pf9fj4e1+Nez7k+d47zPtd3tRACIiIiAJWiLkBERCoOhYKIiORSKIiISC6FgoiI5FIoiIhIrspRF1AajRo1ChkZGVGXISKSUGbNmvVtCKFxQa8ldChkZGQwc+bMqMsQEUkoZrassNfUfCQiIrkUCiIikkuhICIiuRK6T0FEpKxlZ2ezYsUKtm3bFnUppVa9enVatGhBlSpVivw1CgURkXxWrFhBnTp1yMjIwMyiLqfEQgisW7eOFStW0Lp16yJ/nZqPRETy2bZtGw0bNkzoQAAwMxo2bFjsK56UDYXt26OuQEQqqkQPhBwl+RwpFwohwP33Q+fOsH591NWIiFQsKRcK2dkwahQsXgxDh8Lu3VFXJCKyrzVr1jBkyBDatGlD9+7d6dmzJ6+++iqTJk2iXr16HHnkkXTo0IE+ffowZsyYMjtvQoaCmQ0ys5Hff/99sb+2alV4+WVIT4fRo+G+++JQoIhIKYQQOOOMM+jTpw9ffvkls2bNYtSoUaxYsQKA3r17M2fOHBYuXMhDDz3ENddcw4QJE8rk3AkZCiGE0SGEYfXq1SvR12e0Cvz3rgUA3HILTJlSltWJiJTOu+++S9WqVbnyyitzn2vVqhXXXnvtPu/t2rUrv//973n44YfL5NwJGQqlsn07nHkmfW/ozn0/+5xdu+D882HNmqgLE5GKxiw+x4F8+umndOvWrch1duvWjc8//7wUnzRP6oVCtWpQty788APXf3QB/Y7dzqpVMGQI7NoVdXEiIvsaPnw4RxxxBD169Cjw9RBCmZ0r9UIB4OGHoU0bbN5cXut0C02awLvvwh/+EHVhIlKRhBCf40AOO+wwZs+enfv4kUceYcKECWRlZRX4/jlz5tCxY8cy+cypGQp168K//gWVK1N75P2MvW4cZvDHP8K4cVEXJyKp7oQTTmDbtm08+uijuc9t3bq1wPfOnz+fO++8k+HDh5fJuVMzFACOPhruuAOAIx8cyr2/WUsIcOGFEOvgFxGJhJnx2muvMXnyZFq3bs1RRx3F0KFDueeeewCYOnVq7pDU4cOH89BDD9GvX78yOXdqr310443w9tswaRLXz7+U8f3fYNzbxt13ewuTiEhUmjZtyqhRowp8rSTD8Ysqda8UANLS4LnnID0dG/sWT3b9OwDPPw+FXKmJiCS11A4FgBYt4IknAGj+4G+4sPN8vv/eJ7iJiKQahQLAWWfBsGGwYwd/X3cBNdjKyJFRFyUiUv4UCjkeeAAOPZT0bz7jxmoPMX06fPxx1EWJiJQvhUKOmjXhr38F4OrqTwJBVwsiknIUCvn17w9Nm9Lk+yX0YhrPPacOZxFJLQqF/CpXhksuAeDGxs+ow1lEInPXXXdx2GGH0aVLF7p27cqMGTPo27cvM2fOBCAjI4POnTvTpUsX+vfvz+rVq8vkvAqFvV12GQCnbHyJmmzhsccirkdEUs706dMZM2YMs2fPZv78+bzzzju0bNlyn/dNnDiR+fPnk5mZyZ/+9KcyObdCYW8dOkDPnlTZvpmLa/yHDz6A+fOjLkpEUsmqVato1KgR1apVA6BRo0Y0a9as0Pf36dOHJUuWlMm5FQoFiV0tXN/gaQAefzzKYkQkMhGtnd2/f3+WL19O+/btufrqq5k8efJ+3z9mzBg6d+5cJh9ZoVCQwYOhRg3ar5xEa75Uh7OIlKvatWsza9YsRo4cSePGjRk8eDDPPPPMPu87/vjj6dq1Kxs3buTmm28uk3Mn5NpHZjYIGNS2bdv4nKBuXTj7bHj+eW5t/iw/W/kHXn4ZLr00PqcTkQqqDPcpKK60tDT69u1L37596dy5M88+++w+75k4cSKNGjUq0/Mm5JVCabfjLJJYApy37VmM3epwFpFys3DhQhYvXpz7eO7cubRq1apczp2QoVAujj8eWrWizrplnFZrojqcRaTcbN68maFDh9KpUye6dOnCZ599xu23314u507I5qNyUakSDB0Kd9zBb5s/zehF/XjiCXjooagLE5Fk1717d95///19np80aVLu/a+++iou59aVwv7EmpAyl/2HunzPa69F2sQoIhJ3CoX9ad0a+val0vZtDKv7EsuXa5E8EUluCoUDic1Z+Hn1ZwAYMybCWkSkXIQkaRIoyedQKBzI2WdDnTq0XTudDnyuUBBJctWrV2fdunUJHwwhBNatW0f16tWL9XXqaD6QWrXgvPPgySe5vNIz3PTB3WRlQePGURcmIvHQokULVqxYQVZWVtSllFr16tVp0aJFsb7GEjkNMzMzQ86KgXE1aRIcfzzLanUkY8tnPPts7mKqIiIJx8xmhRAyC3pNzUdF0asX1KlDqy0LOIRlakISkaSlUCiKKlXgxBMBOJlxjBsHO3ZEXJOISBwoFIpqwAAAzqszlo0b4b33Iq5HRCQOFApFdfLJABy7/R0qk60mJBFJSgqFomrVCjp2pPqOTfRkukJBRJKSQqE4Yk1IZ1Yfy+LFsGhRxPWIiJQxhUJx5IRCjbGAZjeLSPJRKBRHnz5QowYZ6+dwEKsVCiKSdBQKxVG9OvTtC8AAe5upU2HDhmhLEhEpSwqF4oo1IV3UeCw7d8Lbb0dcj4hIGVIoFNcppwDw481vU4ldakISkaSiUCiutm2hTRtqbl1HJjN5803YtSvqokREyoZCobjMcpuQhqSPZd06mDEj4ppERMqIQqEkYqFwejUNTRWR5KJQKInjj4cqVWi19kMasE6hICJJQ6FQErVrQ+/e2O7dnFbtHT7+GL75JuqiRERKT6FQUvmGpgKMHx9lMSIiZUOhUFKxUPjxxrFA0HwFEUkKCoWSOvxwaNaMWhtX04X5jB8Pu3dHXZSISOkoFEoq39DU8+uNJSsL5s2LuCYRkVKqMKFgZm3M7EkzeyXqWoosFgpn1XgL0JIXIpL44hoKZvaUma01s0/2en6AmS00syVmNgIghPBlCOHyeNZT5k48ESpVol3W+9Rki0JBRBJevK8UngEG5H/CzNKAR4BTgE7ABWbWKc51xEd6OnTrRqVd2fTmPd57D7ZsibooEZGSi2sohBCmAN/t9fRRwJLYlcEOYBRwelG/p5kNM7OZZjYzKyurDKstoX79ALjw4Ans2AGTJ0dcj4hIKUTRp9AcWJ7v8QqguZk1NLN/Akea2c2FfXEIYWQIITOEkNm4ceN413pgsVA4wd4F1K8gIomtctQF5AghrAOujLqOYuvVC6pWpdnq2aTzHW+/3SDqikRESiyKK4WVQMt8j1vEnktMNWtCz55YCJxSfRILFsDy5Qf+MhGRiiiKUPgIaGdmrc2sKnA+8HoEdZSdfP0KoCUvRCRxxXtI6ovAdKCDma0ws8tDCDuBa4BxwALg5RDCp8X8voPMbOT3339f9kWXRCwUev7goTBuXJTFiIiUnIUQoq6hxDIzM8PMmTOjLgOys6FBA9i8measYFuD5qxdC2lpURcmIrIvM5sVQsgs6LUKM6M5oVWpAn36ADC40bt89x3Mnh1xTSIiJaBQKCuxJqRzG3gTkoamikgiUiiUlRNOAOCIdRPQUtoikqgSMhQqXEczQJcu0KgRNdetoL0t4f33YdOmqIsSESmehAyFEMLoEMKwevXqRV1KnkqVfO9m4PKMCezcCZMmRVuSiEhxJWQoVFixfoWBNTQ0VUQSk0KhLMX6FdqvnIixW/0KIpJwFAplqW1baNmSKt+v49ja81i8GJYsibooEZGiUyiUJbPcJqRhbX3V1Ndei7IgEZHiUSiUtVgo9MP7FV59NcpiRESKJyFDoUIOSc0R61c4ePEUalfdwfTpsHp1xDWJiBRRQoZChRySmqNZMzj0UGzLFq7q/iEhwOuJvQasiKSQhAyFCi/WhDS4ifcrqAlJRBKFQiEeYk1InddOoFIlmDABKmJLl4jI3hQK8dC3L5hRdeZ0+h2zhexsePPNqIsSETkwhUI8NGgAPXpAdjbDD30HUBOSiCQGhUK8DBoEwAmbRwPw1luwbVuUBYmIHFhChkKFHpKa47TTAKgz5Q2OPGI3mzd734KISEWWkKFQoYek5jjiCGjZElav5uqjZwFqQhKRii8hQyEhmOVeLQyyMYDPV9i1K8qiRET2T6EQT7FQaPLhaNq0gawsmDYt4ppERPZDoRBPJ5wANWtic+Zw6YkrAC2QJyIVm0IhnqpXh5NOAuCCum8A3q8QQpRFiYgUTqEQb7EmpB99NpqDDoKvvoJ586ItSUSkMAqFeBs4EAB7dwLnDNwKaBSSiFRcCoV4a9rUZzdv28ZlLbXHgohUbAkZCgkxeS2/WBNS1+WjqVMHPv5Y23SKSMV0wFAws1+aWV1zT5rZbDPrXx7FFSYhJq/lF1vyIu2tMZzxk90APP54lAWJiBSsKFcKPw0hbAT6A+nAxcDdca0q2XTtCs2bw6pV3HjSHAAeeww2bYq4LhGRvRQlFCx2eyrwXAjh03zPSVHkm918+NLRHHus76/w1FMR1yUispeihMIsM3sbD4VxZlYH2B3fspJQrAmJ0aO54Qa/++CDWvZCRCqWooTC5cAIoEcIYStQBbgsrlUloxNOgBo1YPZsBnVbSdu2PmdBI5FEpCIpSij0BBaGEDaY2UXAb4EEGfZTgdSoASeeCEDa2Df41a/86fvvj7AmEZG9FCUUHgW2mtkRwA3AF8D/xbWqZJWvCenSSyE9HaZP90NEpCILine truncated
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [ "source": [
"fig = plt.figure().gca()\n", "fig = plt.figure().gca()\n",
"pltx = range(len(loss_gd_y)) \n", "pltx = range(len(loss_gd_y)) \n",
@@ -514,84 +533,67 @@
"fig.plot(pltx, scipy.ndimage.filters.gaussian_filter1d(loss_gd_y,sigma=2) , lw=2, color='blue', label=\"GD\") \n", "fig.plot(pltx, scipy.ndimage.filters.gaussian_filter1d(loss_gd_y,sigma=2) , lw=2, color='blue', label=\"GD\") \n",
"fig.plot(pltx, scipy.ndimage.filters.gaussian_filter1d(loss_sip_y ,sigma=2), lw=2, color='red', label=\"SIP\")\n", "fig.plot(pltx, scipy.ndimage.filters.gaussian_filter1d(loss_sip_y ,sigma=2), lw=2, color='red', label=\"SIP\")\n",
"plt.xlabel('iterations'); plt.ylabel('y loss'); plt.legend(); plt.yscale(\"log\")\n" "plt.xlabel('iterations'); plt.ylabel('y loss'); plt.legend(); plt.yscale(\"log\")\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"id": "KcJemrzNu6dm",
"outputId": "d3f1bf55-4673-44a8-8793-a58c99442753"
},
"execution_count": 26,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3debyWc/7H8den076fNtp0aloUJXVCkxKRSGMXWWKMBjEzmCHMDMOYYRiMYYzsP4wYM4xCSdok0W5JC0ml5UhpU53q+/vjc5+lOqfOdp/r3Pf9fj4e1+Nez7k+d47zPtd3tRACIiIiAJWiLkBERCoOhYKIiORSKIiISC6FgoiI5FIoiIhIrspRF1AajRo1ChkZGVGXISKSUGbNmvVtCKFxQa8ldChkZGQwc+bMqMsQEUkoZrassNfUfCQiIrkUCiIikkuhICIiuRK6T0FEpKxlZ2ezYsUKtm3bFnUppVa9enVatGhBlSpVivw1CgURkXxWrFhBnTp1yMjIwMyiLqfEQgisW7eOFStW0Lp16yJ/nZqPRETy2bZtGw0bNkzoQAAwMxo2bFjsK56UDYXt26OuQEQqqkQPhBwl+RwpFwohwP33Q+fOsH591NWIiFQsKRcK2dkwahQsXgxDh8Lu3VFXJCKyrzVr1jBkyBDatGlD9+7d6dmzJ6+++iqTJk2iXr16HHnkkXTo0IE+ffowZsyYMjtvQoaCmQ0ys5Hff/99sb+2alV4+WVIT4fRo+G+++JQoIhIKYQQOOOMM+jTpw9ffvkls2bNYtSoUaxYsQKA3r17M2fOHBYuXMhDDz3ENddcw4QJE8rk3AkZCiGE0SGEYfXq1SvR12e0Cvz3rgUA3HILTJlSltWJiJTOu+++S9WqVbnyyitzn2vVqhXXXnvtPu/t2rUrv//973n44YfL5NwJGQqlsn07nHkmfW/ozn0/+5xdu+D882HNmqgLE5GKxiw+x4F8+umndOvWrch1duvWjc8//7wUnzRP6oVCtWpQty788APXf3QB/Y7dzqpVMGQI7NoVdXEiIvsaPnw4RxxxBD169Cjw9RBCmZ0r9UIB4OGHoU0bbN5cXut0C02awLvvwh/+EHVhIlKRhBCf40AOO+wwZs+enfv4kUceYcKECWRlZRX4/jlz5tCxY8cy+cypGQp168K//gWVK1N75P2MvW4cZvDHP8K4cVEXJyKp7oQTTmDbtm08+uijuc9t3bq1wPfOnz+fO++8k+HDh5fJuVMzFACOPhruuAOAIx8cyr2/WUsIcOGFEOvgFxGJhJnx2muvMXnyZFq3bs1RRx3F0KFDueeeewCYOnVq7pDU4cOH89BDD9GvX78yOXdqr310443w9tswaRLXz7+U8f3fYNzbxt13ewuTiEhUmjZtyqhRowp8rSTD8Ysqda8UANLS4LnnID0dG/sWT3b9OwDPPw+FXKmJiCS11A4FgBYt4IknAGj+4G+4sPN8vv/eJ7iJiKQahQLAWWfBsGGwYwd/X3cBNdjKyJFRFyUiUv4UCjkeeAAOPZT0bz7jxmoPMX06fPxx1EWJiJQvhUKOmjXhr38F4OrqTwJBVwsiknIUCvn17w9Nm9Lk+yX0YhrPPacOZxFJLQqF/CpXhksuAeDGxs+ow1lEInPXXXdx2GGH0aVLF7p27cqMGTPo27cvM2fOBCAjI4POnTvTpUsX+vfvz+rVq8vkvAqFvV12GQCnbHyJmmzhsccirkdEUs706dMZM2YMs2fPZv78+bzzzju0bNlyn/dNnDiR+fPnk5mZyZ/+9KcyObdCYW8dOkDPnlTZvpmLa/yHDz6A+fOjLkpEUsmqVato1KgR1apVA6BRo0Y0a9as0Pf36dOHJUuWlMm5FQoFiV0tXN/gaQAefzzKYkQkMhGtnd2/f3+WL19O+/btufrqq5k8efJ+3z9mzBg6d+5cJh9ZoVCQwYOhRg3ar5xEa75Uh7OIlKvatWsza9YsRo4cSePGjRk8eDDPPPPMPu87/vjj6dq1Kxs3buTmm28uk3Mn5NpHZjYIGNS2bdv4nKBuXTj7bHj+eW5t/iw/W/kHXn4ZLr00PqcTkQqqDPcpKK60tDT69u1L37596dy5M88+++w+75k4cSKNGjUq0/Mm5JVCabfjLJJYApy37VmM3epwFpFys3DhQhYvXpz7eO7cubRq1apczp2QoVAujj8eWrWizrplnFZrojqcRaTcbN68maFDh9KpUye6dOnCZ599xu23314u507I5qNyUakSDB0Kd9zBb5s/zehF/XjiCXjooagLE5Fk1717d95///19np80aVLu/a+++iou59aVwv7EmpAyl/2HunzPa69F2sQoIhJ3CoX9ad0a+val0vZtDKv7EsuXa5E8EUluCoUDic1Z+Hn1ZwAYMybCWkSkXIQkaRIoyedQKBzI2WdDnTq0XTudDnyuUBBJctWrV2fdunUJHwwhBNatW0f16tWL9XXqaD6QWrXgvPPgySe5vNIz3PTB3WRlQePGURcmIvHQokULVqxYQVZWVtSllFr16tVp0aJFsb7GEjkNMzMzQ86KgXE1aRIcfzzLanUkY8tnPPts7mKqIiIJx8xmhRAyC3pNzUdF0asX1KlDqy0LOIRlakISkaSlUCiKKlXgxBMBOJlxjBsHO3ZEXJOISBwoFIpqwAAAzqszlo0b4b33Iq5HRCQOFApFdfLJABy7/R0qk60mJBFJSgqFomrVCjp2pPqOTfRkukJBRJKSQqE4Yk1IZ1Yfy+LFsGhRxPWIiJQxhUJx5IRCjbGAZjeLSPJRKBRHnz5QowYZ6+dwEKsVCiKSdBQKxVG9OvTtC8AAe5upU2HDhmhLEhEpSwqF4oo1IV3UeCw7d8Lbb0dcj4hIGVIoFNcppwDw481vU4ldakISkaSiUCiutm2hTRtqbl1HJjN5803YtSvqokREyoZCobjMcpuQhqSPZd06mDEj4ppERMqIQqEkYqFwejUNTRWR5KJQKInjj4cqVWi19kMasE6hICJJQ6FQErVrQ+/e2O7dnFbtHT7+GL75JuqiRERKT6FQUvmGpgKMHx9lMSIiZUOhUFKxUPjxxrFA0HwFEUkKCoWSOvxwaNaMWhtX04X5jB8Pu3dHXZSISOkoFEoq39DU8+uNJSsL5s2LuCYRkVKqMKFgZm3M7EkzeyXqWoosFgpn1XgL0JIXIpL44hoKZvaUma01s0/2en6AmS00syVmNgIghPBlCOHyeNZT5k48ESpVol3W+9Rki0JBRBJevK8UngEG5H/CzNKAR4BTgE7ABWbWKc51xEd6OnTrRqVd2fTmPd57D7ZsibooEZGSi2sohBCmAN/t9fRRwJLYlcEOYBRwelG/p5kNM7OZZjYzKyurDKstoX79ALjw4Ans2AGTJ0dcj4hIKUTRp9AcWJ7v8QqguZk1NLN/Akea2c2FfXEIYWQIITOEkNm4ceN413pgsVA4wd4F1K8gIomtctQF5AghrAOujLqOYuvVC6pWpdnq2aTzHW+/3SDqikRESiyKK4WVQMt8j1vEnktMNWtCz55YCJxSfRILFsDy5Qf+MhGRiiiKUPgIaGdmrc2sKnA+8HoEdZSdfP0KoCUvRCRxxXtI6ovAdKCDma0ws8tDCDuBa4BxwALg5RDCp8X8voPMbOT3339f9kWXRCwUev7goTBuXJTFiIiUnIUQoq6hxDIzM8PMmTOjLgOys6FBA9i8measYFuD5qxdC2lpURcmIrIvM5sVQsgs6LUKM6M5oVWpAn36ADC40bt89x3Mnh1xTSIiJaBQKCuxJqRzG3gTkoamikgiUiiUlRNOAOCIdRPQUtoikqgSMhQqXEczQJcu0KgRNdetoL0t4f33YdOmqIsSESmehAyFEMLoEMKwevXqRV1KnkqVfO9m4PKMCezcCZMmRVuSiEhxJWQoVFixfoWBNTQ0VUQSk0KhLMX6FdqvnIixW/0KIpJwFAplqW1baNmSKt+v49ja81i8GJYsibooEZGiUyiUJbPcJqRhbX3V1Ndei7IgEZHiUSiUtVgo9MP7FV59NcpiRESKJyFDoUIOSc0R61c4ePEUalfdwfTpsHp1xDWJiBRRQoZChRySmqNZMzj0UGzLFq7q/iEhwOuJvQasiKSQhAyFCi/WhDS4ifcrqAlJRBKFQiEeYk1InddOoFIlmDABKmJLl4jI3hQK8dC3L5hRdeZ0+h2zhexsePPNqIsSETkwhUI8NGgAPXpAdjbDD30HUBOSiCQGhUK8DBoEwAmbRwPw1luwbVuUBYmIHFhChkKFHpKa47TTAKgz5Q2OPGI3mzd734KISEWWkKFQoYek5jjiCGjZElav5uqjZwFqQhKRii8hQyEhmOVeLQyyMYDPV9i1K8qiRET2T6EQT7FQaPLhaNq0gawsmDYt4ppERPZDoRBPJ5wANWtic+Zw6YkrAC2QJyIVm0IhnqpXh5NOAuCCum8A3q8QQpRFiYgUTqEQb7EmpB99NpqDDoKvvoJ586ItSUSkMAqFeBs4EAB7dwLnDNwKaBSSiFRcCoV4a9rUZzdv28ZlLbXHgohUbAkZCgkxeS2/WBNS1+WjqVMHPv5Y23SKSMV0wFAws1+aWV1zT5rZbDPrXx7FFSYhJq/lF1vyIu2tMZzxk90APP54lAWJiBSsKFcKPw0hbAT6A+nAxcDdca0q2XTtCs2bw6pV3HjSHAAeeww2bYq4LhGRvRQlFCx2eyrwXAjh03zPSVHkm918+NLRHHus76/w1FMR1yUispeihMIsM3sbD4VxZlYH2B3fspJQrAmJ0aO54Qa/++CDWvZCRCqWooTC5cAIoEcIYStQBbgsrlUloxNOgBo1YPZsBnVbSdu2PmdBI5FEpCIpSij0BBaGEDaY2UXAb4EEGfZTgdSoASeeCEDa2Df41a/86fvvj7AmEZG9FCUUHgW2mtkRwA3AF8D/xbWqZJWvCenSSyE9HaZP9Line truncated
},
"metadata": {
"needs_background": "light"
}
}
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "DeSQEpHB0Tip"
},
"source": [ "source": [
"There's likewise an improvements for SIPs, but it is not as pronounced as in $x$ space. Luckily, the $x$ reconstruction is the primary target, and hence of higher importance for the inverse problem at hand.\n", "There's likewise an improvements for SIPs, but it is not as pronounced as in $x$ space. Luckily, the $x$ reconstruction is the primary target, and hence of higher importance for the inverse problem at hand.\n",
"\n", "\n",
"The differences in terms of $L^2$ error are also very obvious in direct comparison. The following cell plots a reconstruction from GD & Adam next to the SIP version, and the ground truth on the right. The difference is obvious: the SIP reconstruction is significantly sharper, and contains fewer halos than the GD version. This is a direct consequence of the undesirable behavior of GD applying diffusion to the backpropagated gradient, instead of working against it." "The differences in terms of $L^2$ error are also very obvious in direct comparison. The following cell plots a reconstruction from GD & Adam next to the SIP version, and the ground truth on the right. The difference is obvious: the SIP reconstruction is significantly sharper, and contains fewer halos than the GD version. This is a direct consequence of the undesirable behavior of GD applying diffusion to the backpropagated gradient, instead of working against it."
], ]
"metadata": {
"id": "DeSQEpHB0Tip"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"source": [ "execution_count": null,
"vis.plot(x_gd.values.batch[0], x_sip.values.batch[0], data.values.batch[0]);"
],
"metadata": { "metadata": {
"id": "gNvf0FuxwfdF",
"colab": { "colab": {
"base_uri": "https://localhost:8080/", "base_uri": "https://localhost:8080/",
"height": 390 "height": 313
}, },
"outputId": "dbea2b17-baf0-43a7-bceb-43921beb4b97" "id": "gNvf0FuxwfdF",
"outputId": "e83e2cd5-a7ee-46d5-a679-637a11c0e822"
}, },
"execution_count": 27,
"outputs": [ "outputs": [
{ {
"output_type": "display_data",
"data": { "data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABCYAAAEoCAYAAACEmfDoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9e7gsWVnm+X4ZedmXc6uioMCqEmgbb0O3oGVhPzIj6HDRQWDUtkFHS8Uu22kv3aPdotMtNq0O2jOjzIAyNYhAj4K0ipQOCojaeEMpRFGBViy5VFlQVJ0697133r7+I/Oc+NYbOyN37h25d2Ts9/c8+ZxcueKyImLFG3HWXu/3mbtDCCGEEEIIIYQQ4ihoHXUDhBBCCCGEEEIIcXzRwIQQQgghhBBCCCGODA1MCCGEEEIIIYQQ4sjQwIQQQgghhBBCCCGODA1MCCGEEEIIIYQQ4sjQwIQQQgghhBBCCCGODA1MiKVjZq81sx8+6nYIIYQQQgixbMzsRjN7l5ldNLP/46jbI8QqoIEJcSDM7HfM7GEz6x11W4QQQszGzJ5qZn9gZufN7KyZ/b6ZfaGZfZOZ/V5Y7iNmtmVml8zsk9PB5RNH2XYhhDgKSA8/sYAe3gHgQQCn3P17ltxMIRqBBibEvjGzxwH4bwE4gOceaWOEEELMxMxOAfg1AP83gOsB3ATg3wHYmbHKV7r7CQCfD+BWAP/mMNophBA15KoePgnAkwF8/x7WeSyAD7i7L7ozM2svuo4QTUADE+IgfCOAdwN4LYDbr/5oZk82sz+ZTl/7BQBroe46M/s1M/vUdKbFr5nZzaH+d8zsh6d/1btkZr9qZo8ws58zswtm9p7pgIgQQoi985kA4O5vcPeRu2+5+9vd/f1lK7n7fQB+HcATD6ORQghRV9z9EwDehskABczsi6bvq+fM7M/M7GnT31+LyXvxv56+y/73ZtYysxeb2d+Y2UNm9iYzu366/OPMzM3sRWb2MQC/Nf39W8zsg9P35beZ2WOvtmW6/D8zs7+e7v+VZmah/p9O171oZh8ws8+f/v5pZvZL0/fwvzWz7zqUkyfEHtDAhDgI3wjg56afZ039dF0AvwLgP2LyV7n/BOCrwzotAD+LyUjypwPYAvAK2u4LAHwDJn/R+wwAfzhd53oAHwTwkiUdjxBCNJW/AjAys9eZ2Zeb2XV7WcnMbgHwFQDet9TWCSFEzZn+Ie3LAXzYzG4C8P8D+GFM3k+/F8Avmdkj3f2bMHk3/nF3P+HuvwngOwE8H8CXAPg0AA8DeCXt4ksAfA4m79TPA/ADAL4KwCMB/C6AN9DyzwHwhQD+IYCvBfCsaTv/MYAfwuQ9/RQms5ofMrMWgF8F8GeYvGN/GYB/YWbPOui5EaIKNDAh9oWZPRWTwYU3uft7AfwNgK8D8EUAOgB+0t0H7v6LAN5zdT13f8jdf8ndr7j7RQA/gokQR37W3f/G3c9j8pe6v3H333T3ISYDHU9e+gEKIUSDcPcLAJ6KifXu/wXwKTO7y8xunLHKr5jZOQC/B+A/A/jRw2mpEELUjl8xs4sAPg7gAUz+QPY/AXiru7/V3cfu/g4Ad2MykLsb/wzA/+ru97r7DiYDB19Dto0fcvfL7r41Xf5/c/cPTt9/fxTAk+KsCQAvc/dz7v4xAL+N6UwOAN+KyaDIe3zCh939o5gMYjzS3V/q7n13vweT58ELDnqChKgCeZjEfrkdwNvd/cFp+eenv90P4D7y1H306hcz2wDwEwCeDeDqX+xOmlnm7qNp+ZNh3a1dygrCJoQQC+LuHwTwTQBgZp8N4P8D8JOYTE1mnj/9K58QQhx3nu/uv2lmX4LJ++4NmPxx7h+b2VeG5TqYDBDsxmMBvNnMxuG3EYA4OPxxWv7llNHDMJnpcPW9+hOh7gry9+NbMPmD4W5t+LTpoPNVMkxmYwhx5GhgQiyMma1jMmUsM7OrotgDcAaTgYmbzMzC4MSnIxfI7wHwWQCe4u6fMLMnYTJF+JovTgghxHJx9w9NfdDfht0HJoQQQgTc/T9PdfN/B/BHAP6ju//TPa7+cQDf4u6/zxUhdprT8j/i7j+3j6Z+HBMr9G6//627P2Ef2xRi6cjKIfbD8zEZ5f1cTKaNPQkTT9zvTuuGAL7LzDpm9lUAbgvrnsRk1sO5adAfxYsQQoglY2afbWbfczXY8DR2xAsxCWAshBBib/wkgGcA+AMAX2lmzzKzzMzWzOxpMaA78SoAP3LVimFmj5zGkZjFqwB8v5n9N9PlT09jR+yFVwP4XjP7Apvw96f7/WMAF83s+8xsfdruJ5rZF+5xu0IsFQ1MiP1wOyZxID7m7p+4+sEkiOULMQnU800AzgL4JwB+Oaz7kwDWMcnt/G4Av3GYDRdCiGPKRQBPAfBHZnYZE/39C0xmsQkhhNgD7v4pAK8H8F0Argao/BQmsxH+FWb/3+rlAO4C8PZpvIp3Y6LJs/bzZgA/BuCNZnYBE73+8j228T9hEsPt5zHR/l8BcP3UMv0cTP6g+LeYvIu/GsDpvWxXiGVj+0ivK4QQQgghhBBCCFEJmjEhhBBCCCGEEEKII0MDE0IIIYQQQgghhDgyNDAhhBBCCCGEEEKII0MDE0IIIYQQQgghhDgy2kfdgL3Q7Wz62tqZ/IdxDNhp6cJURGlsz9mVRlXO2y1bngOKWrqyxzJvt0XLhqEjt/KDs/HMKhi3qey8zF3Xd/8+b7tzziGfp7K65HqUrSeWQ7zOoY9ub59Df3D5QBfkWU/f9IfOjva8/Hvfv/M2d3/2QfYp9k63s+lr3aDHZVf7QLGV974y63VBl0pXDgcwV7v32YYx6+R49rIFPZvzjEueJ3u/9bzF2z3Abbu0PiAOysXLf/eguz/yINtYRJOlx4dPt7Ppa70z8xdsAqxbdYT1Xogp2zt6R57HSgxMrK2dwW1P+p+vlVs7w7xyzotj4WUxWXj2f7htlNYVBgVorokNRrt+BwDvddJyJ5vZ3tFGuuxwI1921Et3ygMGrZ283BqkL77ZTlq2YV62EdXxu3l/mJa3B6HB6bHasORmaZVP0PFO6I5Zuqx3067q7dau34FdXriTyrRYGHQpe6BU+EAsDjIdAnMG7QrnomxT4f4Y9fI++p73/dR+Wpbw0NkR/vhtn77n5bPH/PUNB96p2DNr3TP4oid+27XyQv9ZX4Sy/kj6bOPZ+laAlo16XND9UAcAns0+WF436qZt7aQLb6dlH+Saam16LGdpGwo6GnQz0VCgqFlBd8br6bNm3F3gdaBscIRZZJCowvf5RfRs3yzzPyAlzxt+fsRj5bp3/sG//ehBm7KIJkuPD5+13hnc9qRvP+pmHArjLulhDccpWjt7/0+jOF788Z/+9IG30fR35JUYmBBCHB8cwBgl/7EUQghxaEiThRCiHjRdjzUwIYSoGY4RT3U/AGb2GgDPAfCAuz9xl/p/BeDrp8U2gM8B8Eh3P2tmHwFwEcAIwNDdb62sYUIIsRJIk4UQoh40W49XY2DCkUyZHK3NbnYhNkRSKJ92GS0ABXsATeEdd9L6LEzdap+jabs083a0kbd/sJEey3AjXbh/Ii8PNtM29E+n2x2eCOeoi1KSeBTz4mdQ/8+28hXal9O63rn0HG98Kj8vaw+k5yXbGiTl0WZoNE2R9e5sG0jBFrEkv/OhTA0+KAfQqkWOztfykzxcD9akCuwuk9HgSs/1awG8AsDrd92f+38A8B8AwMy+EsC/dPezYZGnu/uDVTZopckMwxO5DaDVzztdwdZxEMtSSdyFMksFU5hWS6uOgxWJdX2wmU4bHvVCv+/RM4EfS3FTrKEDKgcbXjZgmwqVU2ddYiFpsZ1kmJbjc6pSi0VV2rhId5nTtzxsbF6MqIVYxLayJMse2y+XbQ2UJou68rFn9vLCEdk62hQy4OZ3Xp6xpBAHp+l6vBoDE0KIY4PDMfDqPJru/i4ze9weF38hgDdUtnMhhFhxpMlCCFEPmq7HShcqhKgdY/iePwBuMLO7w+eO/ezTzDYAPBvAL4WfHcDbzey9+92uEEKsOoetx4A0WQghdqPJ78irMWPCHa0Q4TxGD587vX6fWRasz9N/02Xj9GUgjQI/Xk9P64jKcer7aC3d7s5Jmkp8Mq8f9ZIqtGhK78Z9+bLdS+lxdy9Rlo6tvNy5lG4ou9RP93MltWDYpSvXvvvldMra+PJWUvZB2BZHEj99Kim3H/OofDsbqRdlbNRV4yxvTrFawyjNdWBuhoSye4mvXbivrFft+KYDGC02Te3BinzGXwng92mK2lPd/T4zexSAd5jZh9z9XRXsa3VxTy0CpakuF9vuXilk3Sjpny3KkjTukT4HK8fgRNqXd06l5e3r8/0MTqZNGK2VtJ9n+FN1+3K+n2yH69Jy73x67N2Lebl1mXWetP1i2Djdtpw9KrG1VGnRWRKHpvsLWWD2n/K27HjYMtcK94Nn1f+9aUFNrkqPAWmyWIDBIwfzF1oC7ctzvNNCVEjT35E1Y0IIUTsWHA2uiheApqi5+33Tfx8A8GYAt1W5QyGEWAWOQI8BabIQQhRo8juyBiaEELXCAYzc9/ypAjM7DeBLALwl/LZpZievfgfwTAB/UckOhRBiRVhEk6tCmiyEEEWa/o68VCuHmZ0B8GoAT8TkXH4LgP8C4BcAPA7ARwB8rbs/vMx2CCFWiyozNJvZGwA8DROf3b0AXgKgAwDu/qrpYv8jgLe7e5w4fyOAN9tk2nobwM+7+29U2LRDRXoshNgv0uTqkSYLIfZDk/V42TEmXg7gN9z9a8ysC2ADwA8AeKe7v8zMXgzgxQC+b+6WluBpLaTYilZpSsdV8OezxznMPfFOmmKukIIu+Jj7hRSgs1OC9q+juBdnUj/d5snta99PbqSxHtbb6bLnd9aufb/33ImkbnRhMym3tlIzdUyN1Hs4be/6A+mJOvnx3NPcObed1A03U1/eaC0Line truncated
"text/plain": [ "text/plain": [
"<Figure size 864x360 with 6 Axes>" "<Figure size 1080x288 with 6 Axes>"
], ]
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1oAAAFgCAYAAACi3Ah8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9Xaws2XXf9197V3X3vXfu3JE0Js2IckZAhMCwEAoGQSGAEJMQokiGAjmAIYgxDMaRQUCIDT8FkfNgAQ4QJMhD4ECO6EHAUAIiykYSRnyQJRECbBqJP0QGsqEosi0wFDQDiiNyOOTcj9NdtffKw1p7167q6j59zunPU+t30bdPd1dXV1V3rVrfi5gZhmEYhmEYhmEYxv5wp94AwzAMwzAMwzCM+4YZWoZhGIZhGIZhGHvGDC3DMAzDMAzDMIw9Y4aWYRiGYRiGYRjGnjFDyzAMwzAMwzAMY89Up94AwzA6/oOPPOKvvx3utI4v/ovlrzHzD+9pkwzDuKeYvDEM45hMUeaYoWUYZ8TX3w74Z7/2J+60Dv++f/3qnjbHMIx7jMkbwzCOyRRljhlahnFGMICIeOrNMAxjApi8MQzjmExR5liNlmEYhmEYhmEYxp6xiJZhnBWMwNPy9hiGcSpM3hiGcUymJ3PM0DKMM0LC6nzQzyCiTwL4UQBvMfP3jrz+nwP4C/qwAvAnAfwxZn6biL4M4F0AAUDLzB886MYahnEwTN4YhnFMpihzLHXQMM6MeMd/O/ApABs79jDzf8fM38fM3wfgrwP4h8z8drHIR/R1U3oM48IxeWMYxjGZmswxQ8swJgYzfx7A29cuKHwUwKcPuDmGYdxjTN4YhnFMzk3mmKFlGGcEgxH4brd9QUQPIV6h/623icCvE9EXiejje/swwzCOjskbwzCOyRRljtVoGcaZsYf85VeJ6AvF49eZ+fVbrOc/BPB/DkLqP8DMbxLRewB8joh+V71HhmFcICZvDMM4JlOTOWZoGcYZwQDC3YXQ1/aUW/wTGITUmflNvX+LiD4D4EMATPExjAvE5I1hGMdkijLHUgcN48yI4Dvd9gERPQHwZwD8cvHcIyJ6nP4G8EMAfnsvH2gYxkkweWMYxjGZmsyxiJZhTAwi+jSAD0PC728A+BkANQAw8yd0sf8IwK8z87Pire8F8BkiAkR2/CIz/+qxttswjMvD5I1hGMfk3GSOGVqGcUYwsNdiz9HPYP7oDst8CtIitXzuSwA+cJitMgzj2Ji8MQzjmExR5pihZRhnxrRmphuGcUpM3hiGcUymJnPM0DKMM4LB+ygUNQzDuBaTN4ZhHJMpyhxrhmEYhmEYhmEYhrFnLKJlGOcEA2Fazh7DME6FyRvDMI7JBGWOGVqGcUYwppe/bBjGaTB5YxjGMZmizDFDyzDOCkIAnXojDMOYBCZvDMM4JtOTOVajZRiGYRiGYRiGsWcsomUYZwQDiBPLXzYM4zSYvDEM45hMUeaYoWUYZ8bUwuqGYZwOkzeGYRyTqckcM7QM44xgTE8IGYZxGkzeGIZxTKYoc6xGyzAMwzAMwzAMY89YRMswzozI0/L2GIZxOkzeGIZxTKYmc8zQMowzYophdcMwToPJG8MwjskUZY4ZWoZxRjAIwTJ6DcM4AiZvDMM4JlOUOdPaW8MwDMMwDMMwjCNgES3DODOmlr9sGMbpMHljGMYxmZrMMUPLMM6IKeYvG4ZxGkzeGIZxTKYoc8zQMoyzghDYMnoNwzgGJm8Mwzgm05M509pbwzAMwzAMwzCMI2ARLcM4IxhANP+HYRhHwOSNYRjHZIoyxwwtwzgzppa/bBjG6TB5YxjGMZmazDFDyzDOCObp5S8bhnEaTN4YhnFMpihzprW3hmEYhmEYhmEYR8AiWoZxZsSJhdUNwzgdJm8MwzgmU5M5ZmgZxhkhMyYs0GwYxuExeWMYxjGZoswxQ8swzorp5S8bhnEqTN4YhnFMpidzprW3hmEYhmEYhmEYR8AiWoZxRkxxxoRhGKfB5I1hGMdkijLHDC3DODMCT6tQ1DCM02HyxjCMYzI1mWOGlmGcEQyaXKGoYRinweSNYRjHZIoyZ1p7axgGiOiTRPQWEf32htc/TETfJKLf0tvfKF77YSL6l0T0e0T008fbasMwLhGTN4ZhHJNzkzkW0TKMMyMeviPPpwD8LIBf2LLMP2LmHy2fICIP4G8D+PcBvAHgN4nos8z8O4faUMMwDovJG8MwjsnUZI4ZWoZxRhxjxgQzf56IXrvFWz8E4PeY+UsAQES/BODHAJjiYxgXiMkbwzCOyRRljqUOGsYZwSAEvtsNwKtE9IXi9vFbbMq/S0T/nIj+PhH9KX3uOwH8QbHMG/qcYRgXiMkbwzCOyRRljkW0DOPM2EPr068x8wfv8P7/G8C/ycxPiejPAvg/AHzPXTfKMIzzw+SNYRjHZGoyxyJahmH0YOZvMfNT/ftXANRE9CqANwF8V7Ho+/U5wzCMW2HyxjCMY3JsmWMRLcM4I5iBcPhC0a0Q0R8H8FVmZiL6EMQh83UA7wD4HiL6bojw+QkA//HpttQwjLtg8sYwjGMyRZljhpZhnBWEiMMO8yOiTwP4MCTP+Q0APwOgBgBm/gSAPw/gp4ioBfACwE8wMwNoieivAPg1AB7AJ5n5/znoxhqGcUBM3hiGcUymJ3PM0DKMM4JxeG8PM3/0mtd/FtIadey1XwHwK4fYLsMwjovJG8MwjskUZY7VaBmGYRiGYRiGYewZi2gZxplx6BkThmEYCZM3hmEck6nJHDO0DOOMYBAiHzZ/2TAMAzB5YxjGcZmizDFDyzDOjKl5ewzDOB0mbwzDOCZTkznT2lvDMAzDMAzDMIwjYBEtwzgjGEA88YwJwzCmgckbwzCOyRRljhlahnFWEMKBZ0wYhmEIJm8Mwzgm05M5ZmgZxhkxRW+PYRinweSNYRjHZIoyZ1p7axiGYRiGYRiGcQQsomUYZ8bUwuqGYZwOkzeGYRyTqckcM7QM44xgpsmF1Q3DOA0mbwzDOCZTlDlmaBnGmREmJoQMwzgdJm8MwzgmU5M509pbwzAMwzAMwzCMI2ARLcM4IxhAnFj+smEYp8HkjWEYx2SKMscMLcM4K2hyYXXDME6FyRvDMI7J9GSOGVqGcUbIjIlpeXsMwzgNJm8MwzgmU5Q50zIrDcMwDMMwDMMwjoBFtAzjzAjm/zAM40iYvDEM45hMTeaYoWUYZwSDJhdWNwzjNJi8MQzjmExR5pihZRhnRpyYt8cwjNNh8sYwjGMyNZkzrb01DMMwDMMwDMM4AhbRMowzghkIEwurG4ZxGkzeGIZxTKYoc8zQMowzY2r5y4ZhnA6TN4ZhHJOpyRwztAzjjJBCUcvoNQzj8Ji8MQzjmExR5kxrbw3DMAzDMAzDMI6ARbQM48wImFZY3TCM02HyxjCMYzI1mWOGlmGcEYzp5S8bhnEaTN4YhnFMpihzzNAyjLNievnLhmGcCpM3hmEck+nJnGntrWEYhmEYhmEYxhG4iIjWrH7Ei8UrEnNkLl6h0T8ByLLjD3qQvjQWyaTybeXnEg3eQ/1tIFmG89/lZ1Bve4gHm8fpeV57bhTm8e3k4o2b3j8avaX156l7orfP04r+7g6j91u4unoHq+bZzkcr2oE9KbP6ES9mr8iDbV/FtvPyBgtulDOjCxcbxLzh3NzxM5g7OTG8l3cV+z8ma2nD3+PbfSfZsVW+GyVXS5M3l8asfsSL+Sun3ozdyNe2E/xu+BqdxjgJJnO2cxGG1mL+Cj70gZ8CWA2QyIDb4YuKuyk4WQFwBKZOCaA2giKLMdNGILDEAJ0EArlyctP3sCOACHHmECsCO0Kc670HYpWW02XRfX5fEQIocjbCKK5vc34cWV5nwLUMCqIsucCglkHMoCbqcgxqAhABilH2iWX/8rGqPNhTX4FzDqhcd6x02/M+rBmeG7b3OiXymNzmIjHc/kIfHXuOawd2hC/85t++0UccepgfEX0SwI8CeIuZv3fk9b8A4L+A7Nm7AH6Kmf+5vvZlfS4AaJn5gwfd2BOQ5Q0GDg+Fr/ntjL1nI7FwlKRzEZDzGMXz0PPQ9885AIj6OwMR4NDJo0KWyQp02/SeWpVtkbMsoMBAjEAhc0bzHmh47lO3XE9ecfe6yhX2dO0xvPN1+IxEzVEh4J/91s/tvLjJm/NgMX8FH/q+nzr1ZuwGiU7DnhDmDquXPcLscL8hSjoQA/WziNk7K9HLjLPBZM52LsLQylBSNkZeG1NuNhljepGPGnkaKicpfdQFUUIQGH4ZQE3oFAQ1lmLlxLiqHTgZVzWpoQWEGYHVyIo18nPp71gDXImRwh6FZdKFwYbGVX+/O0OMWsAFec6tANeKwuaXgG8YrhVBRS3DLyP8ixaujUAbQSGIAK09uCo0q8hg78T4Ko5dOgZ5MzYpTpsikGPE7S+fnOtke/nbIP0NeNoaaRjjCPnLnwLwswB+YcPr/x+AP8PM3yCiHwHwOoDvL17/CDN/7bCbeEIcIc69Gh1jhtb2t/Pgdz4aNS8Nk0T6/avBQyH2ZBh7NahU9iSZFWZyfrKH3CfZ5KEyDVm2yWdjYHQBFNI9w6mzRp7j7nefou16TEiqmgtDsXtdHGKQfRgewk3Hb8xxcVt2XcctvfJ71ROIbuSEorXMjrth8sa4LeGBw9P3e6we6xMH0J8pqC4TgQdfJdRPHWgV9v9BxtGYmsy5HEOrSMcaXuSIsX7B3HQhKlNYfOeN5RylQTYqOEWUsrJSKEdEogSpd5YdEL0aa15vrjO4JKKlClIFxJmsM86AWHP3vrUUmaHatn5cOkOLwC1EYSMADqBAqhiJdhUrgmPZz6SEkYNYnclTlZQ71mNUeMvXPr487ulP3vD6Foj5dBWDuxp4m35jA+MzRxLKY7kjMszvsN4eZv48Eb225fX/q3j4TwC8/6AbdI7o+cNqiKwZSbvC3Line truncated
}, },
"metadata": { "metadata": {
"needs_background": "light" "needs_background": "light"
},
"output_type": "display_data"
} }
} ],
"source": [
"plt = vis.plot(x_gd.values.batch[0], x_sip.values.batch[0], data.values.batch[0], size=(15,4) )\n",
"plt.get_axes()[0].set_title(\"Adam\"); plt.get_axes()[1].set_title(\"SIP\"); plt.get_axes()[2].set_title(\"Reference\");\n"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "pC5xFvjv7wGz"
},
"source": [ "source": [
"## Next steps\n", "## Next steps\n",
"\n", "\n",
"* For this example, it's worth experimenting with various training parameters: run the training longer, with varying learning rates, and different network sizes (or even different architectures).\n" "* For this example, it's worth experimenting with various training parameters: run the training longer, with varying learning rates, and different network sizes (or even different architectures).\n"
], ]
"metadata": {
"id": "pC5xFvjv7wGz"
}
} }
], ],
"metadata": { "metadata": {
"colab": {
"collapsed_sections": [],
"name": "Heat-SIP-forPBDL.ipynb",
"provenance": []
},
"kernelspec": { "kernelspec": {
"display_name": "Python 3", "display_name": "Python 3",
"language": "python", "language": "python",
@@ -608,11 +610,6 @@
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython2", "pygments_lexer": "ipython2",
"version": "2.7.6" "version": "2.7.6"
},
"colab": {
"name": "Heat-SIP-forPBDL.ipynb",
"provenance": [],
"collapsed_sections": []
} }
}, },
"nbformat": 4, "nbformat": 4,