updated teaser, ref fixes
This commit is contained in:
NT committed
1 parent
08655abcbe
commit
1f8c97972c
2 files changed
+50
-53
No files matched your search
+48
-51
@@ -13,9 +13,9 @@
|
||||
"id": "lesbian-brave",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's directly look at a very reduced example that highlights some of the key capabilities of physics-based learning approaches.\n",
|
||||
"Let's start with a very reduced example that highlights some of the key capabilities of physics-based learning approaches.\n",
|
||||
"\n",
|
||||
"Take a look at the following picture - the desired solution is shown in light gray. If we don't take care we'll learn approximations like the red one shown on the left, which are completely off! With an improved learning setup, ideally by using a discretized numerical solver, we can at least accurately represent a part of the solutions (shown in green on the right).\n",
|
||||
"Take a look at the following picture - the desired solution is shown in light gray. If we don't take care we'll learn an approximation like the red one shown on the left, which is completely off! With an improved learning setup, ideally by using a discretized numerical solver, we can at least accurately represent a part of the solutions (shown in green on the right).\n",
|
||||
"\n",
|
||||
"```{figure} resources/intro-teaser-side-by-side.png\n",
|
||||
"---\n",
|
||||
@@ -39,9 +39,11 @@
|
||||
"id": "funky-tamil",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's illustrate the properties of deep learning via _differentiable physics_ (DP) with a simple example: We'd like to find an unknown function $f^*$ that generates solutions from a space $Y$, taking inputs from $X$, i.e. $f^*: X \\to Y$ (in the following, we'll often denote _idealized_, and unknown functions with a $^*$). \n",
|
||||
"One of the key concepts of the following chapters is what we'll call _differentiable physics_ (DP). This means that we use domain knowledge in the form of model equations, and then integrate discretized versions of these models into the training process. As implied by the name, having differentiable formulations is crucial for this process to support the training of neural networks.\n",
|
||||
"\n",
|
||||
"Let's additionally assume we have a generic differential equation $\\mathcal P^*: Y \\to X$ (our _model_ equation), that encodes a property of the solutions, here modeled in terms of a mapping to the input space $X$. Alternatively, we could imagine $X$ representing future solutions that we're looking for (then $\\mathcal P^*$ would model a time evolution), or it could be a constraint for conservation of mass (then $\\mathcal P^*$ would measure divergence).\n",
|
||||
"Let's illustrate the properties of deep learning via DP with the following example: We'd like to find an unknown function $f^*$ that generates solutions from a space $Y$, taking inputs from $X$, i.e. $f^*: X \\to Y$. In the following, we'll often denote _idealized_, and unknown functions with a $*$ superscript, in contrast to their discretized, realizable counterparts without this superscript. \n",
|
||||
"\n",
|
||||
"Let's additionally assume we have a generic differential equation $\\mathcal P^*: Y \\to Z$ (our _model_ equation), that encodes a property of the solutions, e.g. some real world behavior we'd like to match. Later on, $P^*$ will represent time evolutions, but it could also be a constraint for conservation of mass (then $\\mathcal P^*$ would measure divergence). But to keep things as simple as possible here, the model we'll look at in the following is a mapping back to the input space $X$, i.e. $\\mathcal P^*: Y \\to X$.\n",
|
||||
"\n",
|
||||
"Using a neural network $f$ to learn the unknown and ideal function $f^*$, we could turn to classic _supervised_ training to obtain $f$ by collecting data. This classical setup requires a dataset by sampling $x$ from $X$ and adding the corresponding solutions $y$ from $Y$. We could obtain these, e.g., by classical numerical techniques. Then we train the NN $f$ in the usual way using this dataset. \n",
|
||||
"\n",
|
||||
@@ -63,10 +65,9 @@
|
||||
"id": "latest-amino",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To illustrate these two approaches, we consider the following simplified setting: Given the function $\\mathcal P: y\\to y^2$ for $y$ in the inverval $[0,1]$, find the unknown function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$. Note that the _discretization_ here is simply given by representing the $x$ and $y$ via floating point numbers in the computer.\n",
|
||||
"To illustrate these two approaches, we consider the following simplified setting: Given the function $\\mathcal P: y\\to y^2$ for $y$ in the inverval $[0,1]$, find the unknown function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$. Note: to make things a bit more interesting, we're using $y^2$ here instead of the more common $x^2$ parabola, and the _discretization_ is simply given by representing the $x$ and $y$ via floating point numbers in the computer for this simple case.\n",
|
||||
"\n",
|
||||
"We know possible solutions for $f$ are the positive or negative square root function (for completeness: piecewise combinations would also be possible).\n",
|
||||
"\n",
|
||||
"We can try to solve this by using a neural network to approximate the inverse mapping $f$.\n",
|
||||
"Let's start with the supervised approach."
|
||||
]
|
||||
@@ -88,7 +89,7 @@
|
||||
"id": "numerous-emphasis",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For supervised training, we need a method to find for each datapoint the corresponding solution of the above problem. We simply use our solver $\\mathcal P$ for the problem to pre-compute these solutions: We randomly choose between the positive and the negative square root. This makes sense because in the generic case, when we require optimization techniques to do this step, these methods are not expected to favor one particular mode in multimodal solutions."
|
||||
"For supervised training, we can employ use our solver $\\mathcal P$ for the problem to pre-compute the solutions we need for training: We randomly choose between the positive and the negative square root. This resembles the general case, where we would use optimization techniques to compute this the solutions. These methods are not expected to favor one particular mode in multimodal solutions."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -139,45 +140,41 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "adolescent-yellow",
|
||||
"cell_type": "markdown",
|
||||
"id": "confirmed-cargo",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Loss function\n",
|
||||
"Loss_sv = tf.keras.losses.MeanSquaredError()\n",
|
||||
"model_sv.compile(optimizer='adam', loss=Loss_sv)"
|
||||
"And we can start training via a simple mean squared error loss, using keras `fit` function:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "underlying-continuity",
|
||||
"execution_count": 5,
|
||||
"id": "adolescent-yellow",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
|
||||
"Instructions for updating:\n",
|
||||
"If using Keras pass *_constraint arguments to layers.\n",
|
||||
"Train on 200 samples\n",
|
||||
"Epoch 1/5\n",
|
||||
"200/200 [==============================] - 1s 3ms/sample - loss: 0.4967\n",
|
||||
"40/40 [==============================] - 0s 842us/step - loss: 0.5951\n",
|
||||
"Epoch 2/5\n",
|
||||
"200/200 [==============================] - 0s 222us/sample - loss: 0.4958\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.5594\n",
|
||||
"Epoch 3/5\n",
|
||||
"200/200 [==============================] - 0s 453us/sample - loss: 0.4941\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.5462\n",
|
||||
"Epoch 4/5\n",
|
||||
"200/200 [==============================] - 0s 491us/sample - loss: 0.4933\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.5404\n",
|
||||
"Epoch 5/5\n",
|
||||
"200/200 [==============================] - 0s 493us/sample - loss: 0.4927\n"
|
||||
"40/40 [==============================] - 0s 965us/step - loss: 0.5379\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Loss function\n",
|
||||
"loss_sv = tf.keras.losses.MeanSquaredError()\n",
|
||||
"model_sv.compile(optimizer='adam', loss=loss_sv)\n",
|
||||
"\n",
|
||||
"# Training\n",
|
||||
"results_sv = model_sv.fit(X, Y, epochs=5, batch_size= 5,verbose=1)"
|
||||
]
|
||||
@@ -187,20 +184,20 @@
|
||||
"id": "governmental-mixture",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As both model and data set are very small, the training converges very quickly, but if we inspect the predictions of the network, we can see that it nowhere near the solution we wer hoping to find: it averages between the data points on both sides of the x-axis and therefore, fails to find satisfying solutions to our above problem.\n",
|
||||
"As both model and data set are very small, the training converges very quickly, but if we inspect the predictions of the network, we can see that it nowhere near the solution we were hoping to find: it averages between the data points on both sides of the x-axis and therefore, fails to find satisfying solutions to our above problem.\n",
|
||||
"\n",
|
||||
"The following plots nicely highlights this: it shows the data in blue, and the supervised solution in red. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"execution_count": 6,
|
||||
"id": "sought-basement",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de3xcdZ3/8den06RpWmKTtvZKb0uBAoUWA9JVoSxXheUiIhW5Kciiouu6gvDg54ogirKKq/JYqYoUXFC8LNTbKgIVlKIErVDKpVeg96RJSZvpNf38/jjnpCcnZ5JJmmSSzPv5eMyjM+f6OTPpfOb7/Z7v92vujoiISGcNKnQAIiLSPymBiIhIlyiBiIhIlyiBiIhIlyiBiIhIlyiBiIhIlyiBiLTDzOaa2dpuPN4UM3MzG9xdx+xrzGyNmZ1a6Dik5ymBSJ9nZu80s6fN7E0zqzezP5nZceG6K8zsj4WOUaQYDdhfQTIwmFkF8Evgo8BDQCnwLmBXIePKh5kNdve9hY4jTV+OTfoPlUCkrzsUwN0fdPdmd9/h7r9z9+fNbAbwHWCOmW03s60AZnaWmf3NzBrN7A0zuzk6WKwK6XIze93M6szsptj6oWZ2r5k1mNky4Lh4MGZ2g5mtNLNtZrbMzM6PrbsiLB3daWZbgJvNLGNm/xmeZxVwVnsXm+fxvx2Wxl42s1Ni6xeZ2ZfN7C/htT9iZlWJ677SzF4HHg+Xn2NmL5rZ1nD/GfnEEq7/iJm9FFt/bGz1LDN7Pozzx2ZW1t51Sz/l7nro0WcfQAWwBVgAvBuoTKy/AvhjYtlcYCbBD6SjgU3AeeG6KYAD3wWGAscQlGZmhOtvB54CqoCDgaXA2tixLwTGh8e+CGgCxsVi2Qt8gqB0PxS4Bng5PFYV8ER4/sE5rjef4/8bUBKufxOoCtcvAtYBRwHDgJ8BP0xc933huqEEybkJOC083vXACqA0j1guDM91HGDAIcDkcN0a4C/hvlXAS8A1hf5b0qMH/n8WOgA99OjoAcwA7gXWhl+gC4Ex4bo2CSRl/28Ad4bPoy/SibH1fwHmhc9XAWfG1l0dTyApx14CnBuL5fXE+sfjX57A6e0lkDyOvx6wROyXhs8XAbfH1h0B7AYyseueFlv/OeCh2OtBYVKYm0csvwX+Ncd2a4BLYq+/Cnyn0H9HenT/Q1VY0ue5+0vufoW7TyT4dT2eICmkMrO3m9kTZlZrZm8SlAJGJTbbGHueBYaHz8cDb8TWvZY49mVmtiSs8tkaxhM/dnzfDo+XEntHx1/n4bdy7Hjjc5z/NYKSRa74xsfjcfd94foJecRyMLCynUvJ9f7KAKIEIv2Ku79MUBo5KlqUstkDBKWUg939LQTtJJbnKTYQfDlGJkVPzGwyQdXXtcBIdx9BUMUVP3YynpzHS8rz+BPMLP56EkGpJJI81x6gLkd864HJsfNbuP+6PGJ5A/iHXNcixUEJRPo0MzvczP7dzCaGrw8GPgA8E26yCZhoZqWx3Q4C6t19p5kdD1zciVM+BNxoZpXhOT8RWzeM4Au4NozlQ+xPZO0d75NmNtHMKoEb2tk2n+O/NTxeiZldSFC99+vY+kvM7AgzKwduAX7q7s3txHaWmZ1iZiXAvxO0Bz2dRyzfAz5jZm+zwCFh0pEiogQifd024O3An82siSBxLCX4soOgjeFFYKOZRb+0PwbcYmbbgP8g+KLM1xcIqnVWA78D7o9WuPsy4GvAYoLENRP4UwfH+y5Be8Hfgb8CP8+1YZ7H/zMwnaBUcRvwPnffElt/P0EJbSNQBnyynfO9AlwCfCs83j8D/+zuuzuKxd1/Ep7/AYLP6GGCBnMpIta6OlVE+iozuwK4yt3fmWP9IoK7rr7Xm3FJ8VIJREREukQJREREukRVWCIi0iUqgYiISJcU1WCKo0aN8ilTphQ6DBGRfuW5556rc/fRyeVFlUCmTJlCTU1NocMQEelXzCx1BAVVYYmISJcogYiISJcogYiISJcUVRtImj179rB27Vp27txZ6FCKQllZGRMnTqSkpKTQoYjIASpoAjGze4Czgc3u3mZQunB00P8C3kMwJPQV7v7XcN3lwP8LN/2iuy/oSgxr167loIMOYsqUKbQe5FS6m7uzZcsW1q5dy9SpUwsdjogcoEJXYd0LnNnO+ncTDBw3nWBin/8GCKfp/DzBIHvHA58PRzrttJ07dzJy5Eglj15gZowcOVKlPZEBoqAJxN2fBOrb2eRc4D4PPAOMMLNxwBnAo+5e7+4NwKO0n4japeTRe/Rei/SebDbLunXrWLduHdlsttuP39fbQCbQega1teGyXMvbMLOrCUovTJqUcy4fEZF+L5vN0tTURHNzM9u3b29V2t+6dStTp06lvLy8285X6CqsHufu89292t2rR49u05GyT8hkMsyaNYsjjzySY445hq997Wvs27ev3X3WrFnDAw880OOxXXXVVSxbtqzdbR5++OEOtxGRA5csUWSzWWpra1uer169mk2bNlFXV9emqtjdaWpq6tZ4+noJZB2tp+icGC5bB8xNLF/Ua1F1s6FDh7JkyRIANm/ezMUXX0xjYyNf+MIXcu4TJZCLL+7MZHud973vdTy1xMMPP8zZZ5/NEUcc0aOxiBSz+vp61q/fP3txQ0NDy3MzY8SIEbQ3OK6ZMWzYsG6Nqa+XQBYCl4VTZp4AvOnuGwhmeDs9nHa0Ejg9XNYr4lm/u731rW9l/vz5fPvb38bdWbNmDe9617s49thjOfbYY3n66acBuOGGG3jqqaeYNWsWd955Z87t4tasWcPhhx/OBz/4QWbMmMH73ve+lmt47LHHmD17NjNnzuTDH/4wu3btAmDu3Lktw78MHz6cm266iWOOOYYTTjiBTZs28fTTT7Nw4UKuu+46Zs2axcqVK/nmN7/JEUccwdFHH828efO6/T0SGSjy/S7JZrOtkkdSlDjS2hjLy8uprKzs9uqrlhMX6gE8CGwA9hC0Y1wJXANcE6434C5gJfACUB3b98PAivDxoXzO97a3vc2Tli1b1mZZe5qamnzp0qX+wgsv+NKlS72pqalT+6cZNmxYm2VvectbfOPGjd7U1OQ7duxwd/dXX33Vo2t44okn/KyzzmoVV9p2catXr3bA//jHP7q7+4c+9CG/4447fMeOHT5x4kR/5ZVX3N390ksv9TvvvNPd3U866SR/9tln3d0d8IULF7q7+3XXXee33nqru7tffvnl/pOf/KTlPOPGjfOdO3e6u3tDQ0ObODr7nosMBE1NTb558+aW74zOfJds3rzZX3jhhZyPaP/oHBs2bPDVq1f7li1buiV2oMZTvlMLWoXl7h/oYL0DH8+x7h7gnp6Iqz1NTU0t2d7DOsVuz+oxe/bs4dprr2XJkiVkMhleffXVA9ru4IMP5h3veAcAl1xyCd/85jc57bTTmDp1KoceeigAl19+OXfddRef+tSnWu1bWlrK2WefDcDb3vY2Hn300dRzHH300Xzwgx/kvPPO47zzzuvSdYsMBNlslo0bN7Jr1y6am5uBoJQwderUTn2XDBs2DDNr2f6ggw4iatNtampi2LBhLfv25PdRUl9vA+lz4h9kT9QpAqxatYpMJsNb3/pWvvCFLzBmzBj+/ve/s2/fPsrKylL3ufPOO/PaLlnE7cxttSUlJS3bZzIZ9u7dm7rdr371K5588kl+8YtfcNttt/HCCy8weLD+1GTgie56in+Bx9etWrWqzT5RsujMd0l5eXlL0kmeqzcTRpL+V3dSex9kd6itreWaa67h2muvxcx48803mThxIoMGDWLBggUtv2IOOuggtm3b1rJfru2SXn/9dRYvXsycOXN44IEHeOc738lhhx3GmjVrWLFiBYcccgj3338/J510Ut4xx2PZt28fb7zxBieffDLvfOc7+dGPfsT27dsZMWLEAbwrIn1LNpuloaGBrVu3tiSAZBtDrjueomTR2e+S8vLygiaLNH29Eb1PKi8vZ/To0d32Ye7YsaPlNt5TTz2V008/nc9//vMAfOxjH2PBggUcc8wxvPzyyy2/Uo4++mgymQzHHHMMd955Z87tkg477DDuuusuZsyYQUNDAx/96EcpKyvjBz/4ARdeeCEzZ85k0KBBXHPNNXnHP2/ePO644w5mz57N8uXLueSSS5g5cyazZ8/mk5/8pJKH9DtR43Z9fX2bRu7odtmGhoY2VVBxaf8Ho6QRr27qzu+S3lZUc6JXV1d7ckKpl156iRkzZhQoot61Zs0azj77bJYuXVrQOIrpPZe+KypF7N27l8GDB1NZWUl5eXlLgoh/N8ZLGLW1tWzatKnVsdJKINE5Nm7cyO7duxkxYgRjx47tlWvrbmb2nLtXJ5erCktEik5a+0TUUzveuB2JN3InG7QrKytbkk9SeXk506ZN67kLKTAlkCIyZcqUgpc+RHpSfX09jY2NVFRUUFVVlXO7tPaJtMbtSLyRu6fbQfsTJRAR6ZeiO6AymQzNzc00NzdTV1cHwPbt2wFyJpG09om0xu3o2Gl3PhVz4ogogYhIn5Z2q2xaO0VSY2NjzgQSVS2ltYFE65UgOqYEIiJ9VjxRxBuq09opkioqKtpdryRx4JRARKTXxUsV0LY3dSRXb+1c7RTRhGUdtYFI91AC6QNuu+02HnjgATKZDIMGDeLuu+/m7W9/e6/H8Y//+I+pgzB2xqJFi/jP//xPfvnLX3ZTVNLfRbfLQjDy9I4dO1p1wIuSQNqtsLl6a+fTTiE9TwmkwBYvXswvf/lL/vrXvzJkyBDq6urYvXt3j5wrGgBt0KD0/qMHmjykuKXdAZW8XTY+BDnQqgSRNh5UR0N4KGEUlnqid8XixfDlLwf/HqANGzYwatQohgwZAsCoUaMYP348U6ZMabmjpKamhrlz5wJw8803c+mllzJnzhymT5/Od7/73ZZj3XHHHRx33HEcffTRLT3Z16xZw2GLine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA430lEQVR4nO3deXxdZbno8d/TnZQ0bUOTJpamYyJlKFOBMPSgUA6iKB5aRbSKzFARwevliOCHyxFBtMJVzkE5V6qUQYGCglJFDmChgFKEFiqdhJY00DQtJNlpMzUd0uf+sdbaWXtl7WRnJ9lD8nw/n/3J3mva70phPXnf5x1EVTHGGGP6akSmC2CMMSY3WQAxxhiTEgsgxhhjUmIBxBhjTEosgBhjjEmJBRBjjDEpsQBiTC9EZI6I1A7g9S4Wkb8O1PWyzVC/P9PFAojJCSLyMRF5RUR2ikhURP4mIie4++yBZUwG5GW6AMb0RkSKgD8BXwceA0YCHwd2Z7JcyRCRPFXdl+lyhMnmspncYDUQkwsOAVDVR1S1U1V3qeqzqvqWiBwO/AKYLSKtIrIDQETOFpE3RaRZRLaIyM3exURkuoioiFwkIu+LSIOI3OjbP0pE7heRJhFZD5zgL4yI3CAi74pIi4isF5HP+fZd7NaO7hSRRuBmERkvIkvdsrwGfLSnmxWR34rIdre29ZKIHOHbd7+I/EJEnnO//0URmebbryLyTRGpdu/rDhEZ0UPZDhSRB0WkXkTeE5H/4zv+oyLyvIg0utd6SETG+b5riog84Z7bKCI/D9zH/3V/h5tF5NO9/BubHGQBxOSCd4BOEXlARD4tIsXeDlXdAFwJrFDVMao6zt3VBlwIjAPOBr4uIvMC1/0YcChwBvAfbjAC+B7OQ/6jwKeAiwLnvYtTAzoQ+D7wGxGZ6Nt/ElANTABuA+4GOoCJwKXuqydPAzOAjwBvAA8F9p8P3AqUAqtD9n8OqAKOA+YGvi9Ytp+591EJnIbzO7vEPVaAHwHlwOHAFOBmABGJ4NQK3wOmA5OAJYHvedst4+3AvSIivdy3yTWqai97Zf0L5wF2P1AL7AOWAhPcfRcDf+3l/P8E7nTfTwcUmOzb/xow331fDZzl27cAqO3h2quBub6yvO/bFwH2Aof5tv2wt/L6jh3nlvVA9/P9wBLf/jFAJzDF/ayBsl8FLOuhbHuAmb5tXwOWJyjLPOBN9/1soB7ICznuYmCT73OhW66DMv3fkb0G9mU1EJMTVHWDql6sqpOBI3H+Kv7PRMeLyEki8oLbvLITp5ZSGjhsu+99O87DGPfaW3z73gtc+0IRWS0iO9wmsyMD1/afW4aTa0x4vcC1IyKy0G0iawZq3F2h11fVViDqljns+9/rYV8pkB8oz3s4tQlEZIKILBGRrW5ZfuMrxxTgPU2cQ4n9blW13X07JsGxJkdZADE5R1X/ifOX+JHeppDDHsappUxR1QNx8iTJNqFsw3lAeqZ6b9x8wy+Bq4Hx6jSZrQ1c21+eepwaU+j1QnwFp9npEzhNS9O9r/YdE7uWiIwBSoC6sP3ud/n3+cvWgFM7mhY4fqv7/ofu8UepahHwVV85tgBTRcQ64gxjFkBM1hORw0Tk30Vksvt5CvBl4FX3kA+AySIy0nfaWCCqqh0iciLOgzlZjwHfFZFi9zuv8e0bjfNQrXfLcgldgawbVe0EnsBJWBeKyEy651T8xuL0LmvEafr5YcgxnxGnW/NInFzIq6rqr1lc55Z9CvC/gEd7KNtjwG0iMtYNjtfi1DS8srQCO0VkEnCd7/TXcALtQhEZLSIFInJKD/dlhiALICYXtOAkZf8uIm04gWMt8O/u/ueBdcB2EWlwt10F3CIiLcB/4Dwok/V9nKaczcCzwK+9Haq6HvgJsAIncB0F/K2X612N03yzHafmdF8Pxz7ofvdWYD1dQdLvYZxEfxQ4Hqdm4PcksAonN/MUcG8P33cNToeDauCv7rUXu/u+j5OI3+le5wnvJDf4/BtwMPA+Tm7qSz18jxmCRNUWlDImV4jI/TgJ/f+TYL8CM1R1U1oLZoYlq4EYY4xJiQUQY4wxKbEmLGOMMSmxGogxxpiUDKs+3KWlpTp9+vRMF8MYY3LKqlWrGlS1LLh9WAWQ6dOns3LlykwXwxhjcoqIhM6eYE1YxhhjUmIBxBhjTEosgBhjjEnJsMqBhNm7dy+1tbV0dHRkuijDQkFBAZMnTyY/Pz/TRTHG9NOwDyC1tbWMHTuW6dOnY+vdDC5VpbGxkdraWioqKjJdHGNMP2W0CUtEFovIhyKyNsF+EZG7RGSTiLwlIsf59l0kIhvdV0+zm/aoo6OD8ePHW/BIAxFh/PjxVtszZojIdA7kfuCsHvZ/Gmdpzxk4q8L9PwARKcGZjfQk4ETge/5lTvvKgkf62O/amPRqb2+nvr6e9vb23g/uo4w2YanqSyIyvYdD5gIPqjPfyqsiMs5de3oO8JyqRgFE5DmcQPTIIBfZGGOyTjQapbm5maKiotjn/Px8xo4dy7Zt21BVRISKigoKCwsH7HszXQPpzSTil+Csdbcl2t6NiCwQkZUisrK+vn7QCtofkUiEWbNmccQRR3DMMcfwk5/8hP379/d4Tk1NDQ8//PCgl+3yyy9n/fr1PR7zhz/8oddjjDGDIxqNUldXR2trK3V1ddTV1dHR0UFLSwt1dXXeuvSoKm1tbQP63dkeQPpNVRepapWqVpWVdRuJnxVGjRrF6tWrWbduHc899xxPP/003//+93s8J10B5Fe/+hUzZ87s8RgLIMYMvGSbnpqbm5O6nogwevTogShaTLYHkK3Er+882d2WaHtaDGab4kc+8hEWLVrEz3/+c1SVmpoaPv7xj3Pcccdx3HHH8corrwBwww038PLLLzNr1izuvPPOhMf51dTUcNhhh3H++edz+OGH84UvfCF2D8uWLePYY4/lqKOO4tJLL2X37t0AzJkzJzb9y5gxY7jxxhs55phjOPnkk/nggw945ZVXWLp0Kddddx2zZs3i3Xff5a677mLmzJkcffTRzJ8/f8B/R8YMJd7zJBqNxp4r7e3tbN68mQ8++IDNmzf3+Kzxmq0SKS8vZ8KECQPefAU41ZpMvoDpwNoE+84GngYEOBl4zd1egrPcaLH72gyU9PZdxx9/vAatX7++27aetLW16dq1a3XNmjW6du1abWtr69P5YUaPHt1t24EHHqjbt2/XtrY23bVrl6qqvvPOO+rdwwsvvKBnn312XLnCjvPbvHmzAvrXv/5VVVUvueQSveOOO3TXrl06efJkffvtt1VV9YILLtA777xTVVVPO+00ff3111VVFdClS5eqqup1112nt956q6qqXnTRRfrb3/429j0TJ07Ujo4OVVVtamrqVo6+/s6NGUra2tq0trZWa2trtbGxMfY88V5r167V2trauG0ffvhhj9dsbGzUzZs3a2NjozY2NurGjRu1pqZmQJ5PqqrASg15pmY0iS4ij+AkxEtFpBanZ1U+gKr+Avgz8BlgE9AOXOLui4rIrcDr7qVuUTehPtja2tq6tSkOeFT32bt3L1dffTWrV68mEonwzjvv9Ou4KVOmcMoppwDw1a9+lbvuuoszzzyTiooKDjnkEAAuuugi7r77br71rW/FnTty5Eg++9nPAnD88cfz3HPPhX7H0Ucfzfnnn8+8efOYN29eCndtTG6LRqOxRHZZWVnsGeHVLLxnSFNTU7dzvX0iEkt+99b0VFJSQklJSdzndMh0L6wv97JfgW8k2LcYWDwY5erJ6NGj+/QPm4rq6moikQgf+chH+P73v8+ECRP4xz/+wf79+ykoKAg9584770zquGA32r50q83Pz48dH4lE2LdvX+hxTz31FC+99BJ//OMfue2221izZg15ecN+zKoZgtrb22lra2P06NGxIOEltYFYMruyspLCwsK4P0ATERGKi4spLi7udu1sY/9X91FhYSEVFRWD9g9bX1/PlVdeydVXX42IsHPnTiZPnsyIESN44IEH6OzsBGDs2LG0tLTEzkt0XND777/PihUrmD17Ng8//DAf+9jHOPTQQ6mpqWHTpk0cfPDB/PrXv+a0005Lusz+suzfv58tW7Zw+umn87GPfYwlS5bQ2trKuHHjUv+lGJNBYUHC2+7VJvxdZMOS2l5Lhf8PUE95eTmdnZ1EIhE6OzvjvidbA4fHAkgKCgsLB/QfdteuXcyaNYu9e/eSl5fHBRdcwLXXXgvAVVddxbnnnsuDDz7IWWedFavxHH300UQiEY455hguvvjihMcFHXroodx9991ceumlzJw5k69//esUFBRw3333cd5557Fv3z5OOOEErrzyyqTLP3/+fK644gruuusulixZwmWXXcbOnTtRVb75zW9a8DA5IxqN0tTURF5eHl6vzbAgAYmbs4uKimhtbY27rvf/o/cHqNd0VVxcnPVBoifDak30qqoqDS4otWHDBg4//PAMlSi9ampq+OxnP8vataEzx6TNcPqdm+zmr110dHTEmp48xcXFcXmKCRMmxAJLohoIJM6B5CoRWaWqVcHtVgMxxgx57e3tsUAwatSoWJORf5T2AQccEHpuopxnT83ZwaT2UGUBZBiZPn16xmsfxqSDv2YBJNXzKT8/v9tEn70lswe6OTvXWAAxxgwpwaalcePGJdXzqaysjLFjx8blQHIlmZ0pFkCMMVlty5YttLS0kJ+fz6RJk3p9mAeT20C3nk/etokTJ8b1fCosLBwWTU8DxQKIMSZreE1PXpfW1tbW2ASAu3fvprq6OjamIpHgWC2vGSqYA8nm8RW5wgKIMSYrBEdpJ9Lb7A+JktsWLAZetk+mOCzcdtttHHHEERx99NHMmjWLv//97xkpx7/8y7/0+xrLly+PTXdijF/YpIF+yYzSBpKa/aGwsHBIdJ/NdlYDybAVK1bwpz/9iTfeeIMDDjiAhoYG9uzZMyjf5U2ANmJE+N8NYTP4GpMKf7fLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 432x288 with 1 Axes>"
|
||||
]
|
||||
@@ -248,14 +245,14 @@
|
||||
"id": "acoustic-review",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's apply the differentiable physics approach to find $f$: we'll directly include our discretized model $\\mathcal P$ in the training. \n",
|
||||
"Now let's apply a differentiable physics approach to find $f$: we'll directly include our discretized model $\\mathcal P$ in the training. \n",
|
||||
"\n",
|
||||
"There is no real data generation step; we only need to sample from the $[0,1]$ interval. We'll simply keep the same $x$ locations used in the previous case, and a new instance of a model with the same architecture as before `model_dp`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"execution_count": 13,
|
||||
"id": "extensive-forward",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -276,29 +273,29 @@
|
||||
"id": "conscious-budapest",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The loss function is the **crucial** point now: we directly incorporate the function f into the loss. In this simple case, the `Loss_dp` function simply computes the square of the prediction `y_pred`. \n",
|
||||
"The loss function is the crucial point for training: we directly incorporate the function f into the loss. In this simple case, the `loss_dp` function simply computes the square of the prediction `y_pred`. \n",
|
||||
"\n",
|
||||
"Later on, a lot more could happen here: we could evaluate finite difference stencils on the predicted solution, or compute a whole implicit time-integration step of a solver. Here we effectively have a simple residual equation $y_{\\text{pred}}^2 - y_{\\text{true}} = 0$ which we are minimizing via a _mean-squared error_. It's not necessary to make it so simple: the more knowledge and numerical methods we can incorporate, the better we can guide the training process."
|
||||
"Later on, a lot more could happen here: we could evaluate finite difference stencils on the predicted solution, or compute a whole implicit time-integration step of a solver. Here we have a simple _mean-squared error_ term of the form $|y_{\\text{pred}}^2 - y_{\\text{true}}|^2$, which we are minimizing during training. It's not necessary to make it so simple: the more knowledge and numerical methods we can incorporate, the better we can guide the training process."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"execution_count": 14,
|
||||
"id": "western-leader",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#Loss\n",
|
||||
"mse = tf.keras.losses.MeanSquaredError()\n",
|
||||
"def Loss_dp(y_true, y_pred):\n",
|
||||
"def loss_dp(y_true, y_pred):\n",
|
||||
" return mse(y_true,y_pred**2)\n",
|
||||
"\n",
|
||||
"model_dp.compile(optimizer='adam', loss=Loss_dp)"
|
||||
"model_dp.compile(optimizer='adam', loss=loss_dp)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"execution_count": 15,
|
||||
"id": "artistic-table",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@@ -306,20 +303,16 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
|
||||
"Instructions for updating:\n",
|
||||
"Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
|
||||
"Train on 200 samples\n",
|
||||
"Epoch 1/5\n",
|
||||
"200/200 [==============================] - 0s 1ms/sample - loss: 0.2275\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.2260\n",
|
||||
"Epoch 2/5\n",
|
||||
"200/200 [==============================] - 0s 405us/sample - loss: 0.0520\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.0139\n",
|
||||
"Epoch 3/5\n",
|
||||
"200/200 [==============================] - 0s 338us/sample - loss: 0.0030\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.0015\n",
|
||||
"Epoch 4/5\n",
|
||||
"200/200 [==============================] - 0s 335us/sample - loss: 0.0025\n",
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.0014\n",
|
||||
"Epoch 5/5\n",
|
||||
"200/200 [==============================] - 0s 340us/sample - loss: 0.0024\n"
|
||||
"40/40 [==============================] - 0s 1ms/step - loss: 0.0014\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -338,13 +331,13 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"execution_count": 16,
|
||||
"id": "indonesian-abraham",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deXyU5bnw8d+VSUKYSCAJa2RJcDuilKhRip5qEE6PW8Wq7XFBlmIp7cFW+1rBtxvajVpPta3tUV7FpUXcrWCxelBSrGIlCBTFo2IAgYBkg0AmgSzX+8fzzPhkMpNMQpJJMtfXTz7Ms879TGKu3Pd1L6KqGGOMMe2VFO8CGGOM6Z0sgBhjjOkQCyDGGGM6xAKIMcaYDrEAYowxpkMsgBhjjOkQCyCmXUTkfhH5oWf7myLyqYgcFpFsETlPRD5yt6+IZ1nd8jUrbxvnFonIjVGO5YqIikhy55YQRGSHiEzt5Hu+JCIzO/OevZ2IPCIiP413OfqSTv+fwfReIrIDGAY0AI3AVuAxYImqNgGo6jzP+SnAr4HPq+pmd9+dwH2q+pvuLT2IyCzgRlX91+A+b3kTiapeHO8ymL7PaiAm3JdUdQAwBlgMLAAeinLuMCANeM+zb0zYdsy64q9707vYz0DvYgHERKSqB1V1BfAfwEwROR0+awYQkZOBD9zTD4jIayLyMTAWWOk2YfUTkYEi8pCI7BWRPe61Pvdes0TkDRG5R0QqgEXu/q+JyPsiUiUiL4vImGC53GakeW4z2QER+b04TgXuBya5733AW173daaIvCgiZe69XxSRkWGPfoKIvC0i1SLygohkRfp8WnuuCOcuEpFnRORJETkkIu+IyISw0/JF5J8ictA9L8299l0R+ZLnXikiUi4iZ4hImoj8SUQq3M9ivYgMc89r1hwnIl93P9NDIrJVRM509y9wy39IRD4QkSlRnuFSEdnofi67RGSR51iweW+uiJS6n8mtsT6/24S3QET+CdSISLKInOo+wwEReU9ELo+lLO7xfxWRN91rd7k106BMEfmLW45/iMgJkZ7XxEhV7cu+UFWAHcDUCPs/Ab7pvn4E+Kn7OhdQIDnaPYDngQeAdGAo8DbwDffYLJzmsptwmlP7A9OAbcCp7r4fAG967qfAi8AgYDRQBlzkud/fw8ruLW82cBXgBwYATwN/9pxbBOwBTnfL+yzwp0jP2tpzRfj8FgH1wNVACnArsB1I8XxmbwM5QBbwPjDPPXYb8KTnXtOALe7rbwAr3efxAWcBGZ5nudF9/RX3uc4GBDgRp6Z4CrALyPE84wlRnqEQGI/zR+fngE+BK8I+m+Xu5zHe/b5MbcfzbwJGuT8DKe7PwP8FUoELgUPAKTGUZYx77rXufbKBfM/PQgVwDs7P1jLgiXj/f9ebv6wGYmJRivOLrV3cv4YvAW5W1RpV3Q/cA1zjvbeq/k5VG1S1FpgH/EJV31fVBuDnOH+dj/Fcs1hVD6jqJ8AaID+W8qhqhao+q6oBVT0E/Ay4IOy0P6rqu6paA/wQ+Gp4zSLG5wq3QVWfUdV6nLxRGvB5z/HfqmqpqlbiBIXgM/0JuEREMtztG4A/uq/rcX5Bnqiqjaq6QVWrI7z3jcBdqrpeHdtUdSdOnqsfME5EUlR1h6p+HOWzK1LVLarapKr/xAkW4Z/dHe7nsQV4GOeXeHuef5f7M/B54Dic7/NRVX0N54+Ga2Moy3XAalVdrqr17vd8k+d9nlfVt92frWXE+LNjIrMAYmJxPFDZgevG4PwVuNdtTjiA81f7UM85uyJc8xvP+ZU4fzUf7zlnn+d1AOeXTZtExC8iD4jIThGpBtYCg8IChLc8O93yD+7Ac4UL3VedDgm7cWocrT6TqpYCbwBXicgg4GKcX3zgBJKXgSfcpqO7xOnYEG4U0CIwqOo24GacGsJ+EXlCRHLCzwMQkYkissZt/juIE+jDP5fwzy4n0rEoz++9NgfY5Z7nvd/xMZQl4rN6dOhnx0RmAcS0SkTOxvkf9+8duHwXcAQYrKqD3K8MVT3Nc074dNC7cJqCBnm++qvqmzG8X1tTS/8fnGabiaqaAZzv7hfPOaM8r0fj/JVf3oHnChe6r4gkASNxanaxeBSYjtMUtU5V9wC4f2HfoarjgHOBy4AZEa7fBURs61fVx9XptTYG5/P7ZZQyPA6sAEap6kCcfJOEnRP+2ZVGOhbl+b3fu1JglHue9357YihL1Gc1nc8CiIlIRDJE5DLgCZw8wJb23kNV9wKvAP/l3i9JRE4QkfCmD6/7gdtF5DS3HANF5CsxvuWnwEgRSY1yfABQi5P0zwJ+HOGc6SIyTkT8wJ3AM6ra2AnPdZaIXClOL6ObcQLQWzE+15+BM4Hv4HSrBkBEJovIeLcGVY0T7JoiXP8gcKuInCWOE0VkjIicIiIXikg/oA7ns4l0PTifXaWq1onIOThNReF+6NbyTgNmA0928Pn/gVM7uE2cTgOFwJdwfhbbKssyYKqIfNVNxmeLiDVTdRELICbcShE5hPOX3Pdx2qtnH8P9ZuAkQrcCVcAzwIhoJ6vq8zh/BT/hNjO9i9NsE4vXcLoQ7xOR8FoDwL04SdpynF9ef41wzh9xkq37cNrpvx3lvdr1XMALOD3aqnDyGFe6+YA2uXmBZ4E84DnPoeHu+1bjJN7/xmf5Ee/1T+Pkex7HSTD/GSen1Q+nq3Y5zvMOBW6PUoxvAXe6Pxs/Ap6KcM7fcJLfrwJ3q+ornmMxP7+qHsUJGBe7ZfsDMENV/7etsrh5sUtwapuVOMn58B5vppOIqi0oZUxXcruZnqiq04/hHj8CTj6We3QVEcnls15VDRGOL+IYn9/0TDZox5gezm1um4Pzl7sxPYY1YRnTg4nI13GaE19S1bXxLo8xXtaEZYwxpkOsBmKMMaZDEioHMnjwYM3NzY13MYwxplfZsGFDuaoOCd+fUAEkNzeX4uLieBfDGGN6FRHZGWm/NWEZY4zpEAsgxhhjOsQCiDHGmA5JqBxIJPX19ezevZu6urp4F6XPSktLY+TIkaSkRJoo1hjTWyV8ANm9ezcDBgwgNzcXkfDJRc2xUlUqKirYvXs3eXl58S6OMaYTJXwTVl1dHdnZ2RY8uoiIkJ2dbTU8Y/qghA8ggAWPLmafrzHxs27XOn7x+i9Yt2tdp9874ZuwjDGmr1mwegGP//Nx+qf0p6SqBEXp5+vHqzNeZdKoSZ32PlYD6QF8Ph/5+fmcdtppTJgwgf/6r/+iqSnauj6OHTt28Pjjj3dJeS655BIOHDjQ6jmPPPIIpaWxLqhnjOkOSzYsIe/ePO564y52H9rNR5Uf0aiNNGkTRxqPULSjqFPfL641EBFZirMM535VPT3CcQF+g7NATACYparvuMdmAj9wT/2pqj7aPaXufP3792fTpk0A7N+/n+uuu47q6mruuOOOqNcEA8h110VaGO7YrFq1qs1zHnnkEU4//XRyciIuoW2M6SLrdq2jaEcRhbmFAKHXW/Zv4RsvfiPqdT7xha7pLPGugTwCXNTK8YuBk9yvucB/Q2h9hB8DE4FzgB+LSGaXltQjEAhQVlZGIBDo9HsPHTqUJUuWcN9996Gq7Nixgy984QuceeaZnHnmmbz5prM0+MKFC3n99dfJz8/nnnvuiXpeUVER559/PpdeeimnnHIK8+bNC9Vuli9fzvjx4zn99NNZsGBBqAy5ubmUl5ezY8cOTj31VL7+9a9z2mmn8cUvfpHa2lqeeeYZiouLuf7668nPz6e2tpaFCxcybtw4Pve5z3Hrrbd2+udiTKJbt2sdX37iy3zh4S/wgzU/YPKjkyl8tJAfrvkhUx6bwkPvPBT1Wp/4uO+S+zq1+QpwulnG8wvIBd6NcuwB4FrP9gc4y4ZeCzwQ7bxoX2eddZaG27p1a4t9rampqdF3331Xt2zZou+++67W1NS06/pI0tPTW+wbOHCg7tu3T2tqarS2tlZVVT/88EMNPsOaNWv00ksvbVauaOf169dPP/74Y21oaNCpU6fq008/rXv27NFRo0bp/v37tb6+XidPnqzPP/+8qqqOGTNGy8rKdPv27erz+XTjxo2qqvqVr3xF//jHP6qq6gUXXKDr169XVdXy8nI9+eSTtampSVVVq6qqWjxPez9nY8xn3vzkTe33k37KIkJfskhUFomyCPXd4dMrll/R7HjGzzL0tN+epjc8cYO+sOGFY/pdBRRrhN+pPT2JfjzOYjpBu9190fa3ICJzcWovjB49+pgLVFNTEwxaqCo1NTX4/f5jvm809fX1zJ8/n02bNuHz+fjwww/bfd4555zD2LFjAbj22mv5+9//TkpKCoWFhQwZ4kywef3117N27VquuOKKZvfNy8sjPz8fgLPOOosdO3a0eO+BAweSlpbGnDlzuOyyy7jssss649GNMa6iHUUcbTzabF+SJOFL8tHY1EiqL5XbzruNKaOn8NR7TzF52GS+MvYrzc7fvn07eXl5nfr7qqcHkGOmqkuAJQAFBQXHvHpWeno6IoKqIiKkp6cfcxnDlZSU4PP5GDp0KHfccQfDhg1j8+bNNDU1kZaWFvGae+65J+p54d1o29Ottl+/fqHXPp+P2traFuckJyfz9ttv8+qrr/LMM89w33338dprr8X8HsYkmmAeI9ufTUWggsLcwlablwpzC0n1pXKk8QjgNEn9qvBXDGUo6/ev5+yhZ3NK+ilkDMygcFJhxHt0xR+8PT2A7AFGebZHuvv2AIVh+4u6o0B+v5+8vDxqampIT0/v9NpHWVkZ8+bNY/78+YgIBw8eZOTIkSQlJfHoo4/S2NgIwIABAzh06FDoumjnAbz99tts376dMWPG8OSTTzJ37lzOOeccvv3tb1NeXk5mZibLly/npptuirmc3vc/fPgwgUCASy65hPPOOy9U2zEm0a3btY6FqxdSUlXCBbkXcNqQ08j2Z3PzX2/mSOMRmrSJJJLLine truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7+0lEQVR4nO3deXxU9bn48c+TSSAECSQQZIekooJS0UYrequo1HrRioq1bkUsFdFLq/S2irX3V+ttK/W2Um9pq1RR0Nalrrj0akUjWoMaBIpCrQhhC0tCEgKZrJPn98c5M5yZzCSTdbI879crL87yPed8zwB55ruLqmKMMca0VFKiM2CMMaZ7sgBijDGmVSyAGGOMaRULIMYYY1rFAogxxphWsQBijDGmVSyAmBYTkQdE5L88+zeJyD4ROSwig0XkTBH5zN2/JIFZDebvExGZGmdaFZFjYpybLSLvtmfe3PuOc5+b3M73PSwiOe15z+6soz7n3sw+SBNGRAqBo4F6IABsAlYAS1W1AUBV53nSpwD3Aaer6gb32N3AElW9v3NzDyLyKLBLVX8cPKaqJ3R2ProCVT0q0XkwPZuVQEw0X1fVAcBYYBFwO/BwjLRHA6nAJ55jYyP242bfDo39G+g+LICYmFT1oKquBL4JXCciJ4LzLV9EfiYixwKfusnLReRNEfkcyAFecqtQ+orIQBF5WET2iMhu91qfe6/ZIvJ3EVksIgeAu9xrfiUiO9yqsQdEpJ+bfqqI7BKR/xSR/e49r3fPzQWuAW5zn/2Se7xQRKa526eJSL6IlLvXLhGRPhGvPl1EtopIiYj8j4hE/X8iIseLyN9EpFREPhWRK2J9liKSJyL3iMgHIlIhIi+KSGZEsmvcdy4RkTvd64aJiF9EBnvudYqIFItIiogcIyJvi8hB97qnPOlC1XEi0k9Efi0i292077rHUkXkcRE54H4mH4rI0THeYaGIfC4ih0Rkk4hc6jkX/Htc4t7/nyJyXjzv76lamiMiO4A3RSRJRH7s5ne/iKwQkYGe+/1FRPa6z1otIid4zkV916Y+Z9NKqmo/9hP6AQqBaVGO7wBucrcfBX7mbo8DFEiOdQ/geeBBoD8wFPgAuNE9Nxunuuy7OFWq/YDFwEogExgAvATc46af6qa/G0gBpgN+ICMyb9HyA3wJON191jhgM3CrJ60Cb7nPHgP8C/iOJ6/vutv9gZ3A9e69TgZKgIkxPtc8YDdwonvts8DjEZ/hH933PwmoASa4518Nfvbu/mLgt+72E8CdOF8GU4F/i3iXY9zt37l5GAn4gDOAvsCN7ueb5h7/EpAe4x2+AYxwn/VNoBIYHvH3uMD9e/kmcBDIbMH7r3DP9QO+DWzB+TJyFPAc8JgnL9/G+bfRF/gNsN5zLta7Nvk5208rfl8kOgP207V+iB1A1gB3utuPEmcAwaniqgH6ec5fBbzlbs8GdnjOifuL6QueY1OAbe72VKAq4nn7cdpgwvLW3Du5524FnvfsK3CBZ/9mYJUnr8EA8k3gnYh7PQj8JMZz8oBFnv2JQK37Cy74GY7ynP8AuNLzrL+72z5gL3Cau78CWOq9NuJdjsH5hV8FnBQlzbeB94AvtuLfynpghuezKQIk4h2+1YL3z/GcXwXc7Nk/Dqjz/r17zg1yrx/YzLs2+TnbT8t/rK7RxGskUNqK68bifCPdIyLBY0k4396DvNtZON+G13rSC84vmqADqlrv2ffjfEttljjVbvcBue5zkoG1Ecm8+dmO86070ljgyyJS7jmWDDzWxOMj75sCDPEc2+vZ9r7Ti8ADIpKN84v0oKp+4J67Dfhv4AMRKQN+rarLIp47BKd08nmUPD0GjAaeFJFBwOM4XxTqIhOKyCzg+zi/iHHz583/bnV/K3ve0fvZNff+3vMj3DTe9MnA0SKyF/g5TokoC2jwvGffJt41KNbnbFrI2kBMs0TkVJwA0pourDtxSiBDVHWQ+5Ou4T2jvL90SnC+QZ7gST9Q4+9R1Nz00n8A/gmMV9V04Ec4AcprtGd7DM4360g7gbc9eRykqkep6k1NPDvyvnU479skVa0GngauBb6FJ0ip6l5VvUFVR+BUR/1eGndDLgGqgS9EuXedqv5UVSfiVPVcBMyKTCciY3GqfuYDg1V1EPAx4Z/dSPFEfRp/ds29v/fvrggnSHvT1wP7gKuBGcA0nFLHuGA2m3pX0/4sgJiYRCRdRC4CnsSpr97Y0nuo6h7gdeDX7v2SROQLInJ2jPQNOL+oFovIUDcfI0Xka3E+ch9OvXksA4AK4LCIHA9E+4X/QxHJEJHRwC3AU1HSvAwcKyLfchuzU0TkVBGZ0MSzrxWRiSKShtOG84yqBuJ6K6eqajZwMZ4AIiLfEJFR7m4Zzi/hBu+F7me6DLhPREaIiE9EpojTWeEcEZkkTqeGCpxf6mHXu/q79y52n3s9TnuG11Dge+5n8Q1gAk77TWve/wlggYhki8hRwC+Ap9yS5wCcLyUHcEqRv4jnXWM8x7SBBRATzUsicgjnW/adOFU+17fhfrOAPjhjSsqAZ4DhTaS/HacBdY2IVABv4FTdxONhYKLbo+iFKOd/gPMN9hBOoIoWHF7EqdZaD7xClC7MqnoIOB+4Eufb8l7glzhVKLE8htNGsxenmuV7cbxP8Hl/x/nF/pGqeqt2TgXeF5HDOB0PblHVrVFu8QNgI/AhTlXkL3H+/w/D+fuowOlQ8DZRquFUdRPwayAfJ0hPAv4ekex9YDxOKeDnwOWqesBzviXvv8xNvxrYhlOq+K57bgVOldZunH9Ta+J8V9POJLzK0hjTEUQkD6cU91Ab7vEm8Oe23KOjiMhsnN5q/xbjfB5tfH/T9VgjujHdgNsOdQpO3b8xXYIV64zp4kRkOU413q1u1ZkxXYJVYRljjGkVK4EYY4xplV7VBjJkyBAdN25corNhjDHdytq1a0tUNSvyeK8KIOPGjaOgoCDR2TDGmG5FRLZHO25VWMYYY1rFAogxxphWsQBijDGmVXpVG0g0dXV17Nq1i+rq6kRnpcdKTU1l1KhRpKSkJDorxph21OsDyK5duxgwYADjxo0jfCJR0x5UlQMHDrBr1y6ys7MTnR1jTDvq9VVY1dXVDB482IJHBxERBg8ebCU8Y3qgXh9AAAseHcw+X2MSJ39nPve8cw/5O/Pb/d4JrcISkWU4C9jsV9XItQVwF6e5nyPrXs9W1Y/cc9cBP3aT/kxVl3dOro0xputaunYp97x7D2VVZYwbNI7NJZsJNATo4+vDqlmrmDJ6Srs9K9ElkEeBC5o4/+846wuMB+birCaHiGQCPwG+DJwG/EREMjo0px3I5/MxefJkTjjhBE466SR+/etf09AQbU2fIwoLC/nzn//cIfmZPn065eXlTaZ59NFHKSqKtlCfMaazLV27lK899jWufe5abnz5RgrLCzlYc5AN+zZQG6gloAFqA7XkFea163MTWgJR1dUiMq6JJDOAFe46y2tEZJCIDAemAn9T1VIAEfkbTiB6ooOz3CH69evH+vXrAdi/fz9XX301FRUV/PSnP415TTCAXH311e2en1dffbXZNI8++ignnngiI0ZEWy7cGNPe8nfms/CNhWwu2czI9JGcPvJ0Zp00i437N3Ljyzc2ea0gpCSlcPqw09s1T4kugTRnJM6qeEG73GOxjjciInNFpEBECoqLi9slU36/n+LiYvx+f7vcz2vo0KEsXbqUJUuWoKoUFhbyla98hVNOOYVTTjmF9957D4CFCxfyzjvvMHnyZBYvXhwzXV5eHmeddRYXXnghxx13HPPmzQuVbp544gkmTZrEiSeeyO233x7Kw7hx4ygpKaGwsJAJEyZwww03cMIJJ3D++edTVVXFM888Q0FBAddccw2TJ0+mqqqKhQsXMnHiRL74xS/ygx/8oN0/F2N6m/yd+dz08k3c9PJNLF27lK888hVW71hNsb+Y9XvX88DaB5i6fCoPf9RowcwwSZLEN7K/wR+/8kdOHNSopaBNenw3XlVdCiwFyM3NbfPc9X6/n23btqGqiAjZ2dmkpaW1OZ9eOTk5BAIB9u/fz9ChQ/nb3/5Gamoqn332GVdddRUFBQUsWrSIX/3qV7z88suhfEVLB/DBBx+wadMmxo4dywUXXMBzzz3HGWecwe23387atWvJyMjg/PPP54UXXuCSSy4Jy8tnn33GE088wR//+EeuuOIKnn32Wa699lqWLFnCr371K3Jzczlw4ADPP/88//znPxGRZqu/jDFH5O/MJ68wj/KactbvWc/MiTOZNHQS5yw/h5pADQA+8RGIsnx8XaCOIalDwo5dOPpC1h1Yx6HaQxw76FhuPfFWJg+ejIjQv3//ds17Vw8gu4HRnv1R7rHdONVY3uN5nZGhyspKgmuoqCqVlZXtHkC86urqmD9/PuvXr8fn8/Gvf/2rxelOO+00cnJyALjqqqt49913SUlJYerUqWRlORNsXnPNNaxevbpRAMnOzmby5MkAfOlLX6KwsLDRswcOHEhqaipz5szhoosu4qKLLmr7ixvTQ+XvzGfFhhUAnDz8ZG79v1uprq9GcX6vvL71db5+zNepDdSGrmnQBgQJpQlK8aUw57g55A7M5Y3dbzBt5DS+kfONsDQjRowgEAjQv3//dv9d1dUDyEpgvog8idNgflBV94jIa8AvPA3n5wN3dEaG+vfvj4iESiDtHdEBtm7dis/nY+jQofz0pz/l6KOPZsOGDTQ0NJCamhr1msWLF8dMF9mNtiXdavv27Rva9vl8VFVVNUqTnJzMBx98wKpVq3jmmWdYsmQJb775ZtzPMKY3CAaOhz56iHqtB5yShao2Cgw7D+4kJSmF2gYniKT4Uvjtv/+WP/3jT43aQI7rfxzHph0bChwDBw6kpqaGlJQUsrKyOvQLbqK78T6BU5IYIiK7cHpWpQCo6gPAqzhdeLfgdOO93j1XKiL/DXzo3uruYIN6R0tLSyM7O5vLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 432x288 with 1 Axes>"
|
||||
]
|
||||
@@ -390,9 +383,9 @@
|
||||
"source": [
|
||||
"## Discussion\n",
|
||||
"\n",
|
||||
"It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and often it's not clear where (and how many) exist in the solution manifold we're interested in. Using supervised learning is very dangerous in such cases - we might simply and unknowingly _blur_ out these different modes.\n",
|
||||
"It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and it's often not clear where (and how many) exist in the solution manifold we're interested in. Using supervised learning is very dangerous in such cases - we might simply and unknowingly _blur_ out these different modes.\n",
|
||||
"\n",
|
||||
"A good examples are bifurcations in fluid flows - the smoke rising above a candle will start out straight, and then, due to tiny perturbations in its motion, start oscillating in a random direction. The images below illustrate this case via _numerical perturbations_: the perfectly symmetric setup will start turning left or right, depending on how the approximation errors build up. Similarly, we'll have different modes in all our numerical solutions, and typically it's important to recover them, rather than averaging out. Hence, we'll show how to leverage training via _differentiable physics_ in the following chapters for more practical and complex cases.\n",
|
||||
"A good and obvious example are bifurcations in fluid flows - the smoke rising above a candle will start out straight, and then, due to tiny perturbations in its motion, start oscillating in a random direction. The images below illustrate this case via _numerical perturbations_: the perfectly symmetric setup will start turning left or right, depending on how the approximation errors build up. Similarly, we'll have different modes in all our numerical solutions, and typically it's important to recover them, rather than averaging them out. Hence, we'll show how to leverage training via _differentiable physics_ in the following chapters for more practical and complex cases.\n",
|
||||
"\n",
|
||||
"```{figure} resources/intro-fluid-bifurcation.jpg\n",
|
||||
"---\n",
|
||||
@@ -410,9 +403,13 @@
|
||||
"source": [
|
||||
"## Next steps\n",
|
||||
"\n",
|
||||
"- The example above is itentionally using a fairly \"bad\" setup. Change the code above to obtain a solution such as the green one shown in the very first image at the top.\n",
|
||||
"For each of the following notebooks, there's a \"next steps\" section like the one below which contains recommendations about where to start modifying the code. After all, the whole point of these notebooks is to have readily executable programs as a basis for own experiments. The data set and NN sizes of the examples are often quite small to reduce the runtime of the notebooks, but they're nonetheless good starting points for potentially complex and large projects.\n",
|
||||
"\n",
|
||||
"- Or you can try extending the setup to a 2D case, i.e. a paraboloid. Given the function $\\mathcal P:(y_1,y_2)\\to y_1^2+y_2^2$, find an inverse function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$."
|
||||
"For the simple DP example above:\n",
|
||||
"\n",
|
||||
"- This notebook is itentionally using a very simple setup. Change the training setup and NN above to obtain a higher-quality solution such as the green one shown in the very first image at the top. \n",
|
||||
"\n",
|
||||
"- Or try extending the setup to a 2D case, i.e. a paraboloid. Given the function $\\mathcal P:(y_1,y_2)\\to y_1^2+y_2^2$, find an inverse function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
+2
-2
@@ -34,7 +34,7 @@
|
||||
|
||||
@article{um2020sol,
|
||||
title={Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers},
|
||||
author={Um, Kiwon and Brand, Robert and Holl, Philipp and Fei, Raymond Thuerey, Nils},
|
||||
author={Um, Kiwon and Brand, Robert and Holl, Philipp and Fei, Raymond and Thuerey, Nils},
|
||||
journal=NeurIPS,
|
||||
year={2020},
|
||||
url={https://ge.in.tum.de/publications/2020-um-solver-in-the-loop/},
|
||||
@@ -79,7 +79,7 @@
|
||||
|
||||
@inproceedings{holl2019pdecontrol,
|
||||
title={Learning to Control PDEs with Differentiable Physics},
|
||||
author={Holl, Philipp and Thuerey, Nils and Koltun, Vladlen},
|
||||
author={Holl, Philipp and Koltun, Vladlen and Thuerey, Nils},
|
||||
booktitle=ICLR,
|
||||
year={2019},
|
||||
url={https://ge.in.tum.de/publications/2020-iclr-holl/},
|
||||
|
||||
Reference in new issue
Block a user