upd intro parabola, cleanup

_toc.yml

@@ -1,6 +1,8 @@
 # PBDL Table of content (cf https://jupyterbook.org/customize/toc.html)
 #
-- file: intro
+- file: intro.md
+  sections:
+    - file: intro-teaser.ipynb
 - file: overview.md
   sections:
     - file: overview-equations.md
@@ -21,6 +23,7 @@
     - file: diffphys-discuss.md
     - file: diffphys-code-ns.ipynb
     - file: diffphys-code-sol.ipynb
+    - file: diffphys-outlook.md
 - file: jupyter-book-reference
   sections:
     - file: markdown

diffphys-code-ns.ipynb

@@ -9,15 +9,16 @@
     "# Differentiable Fluid Simulations\n",
     "\n",
     "Next, we'll target a more complex example with the Navier-Stokes equations as model. We'll target a 2D case with velocity $\\mathbf{u}$, no explicit viscosity term, and a marker density $d$ that drives a simple Boussinesq buoyancy term $\\eta d$ adding a force along the y dimension:\n",
-    "$\n",
+    "\n",
-    "  \\frac{\\partial u_x}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla u_x = - \\frac{1}{\\rho} \\nabla p \n",
+    "$\\begin{aligned}\n",
+    "  \\frac{\\partial u_x}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla u_x &= - \\frac{1}{\\rho} \\nabla p \n",
     "  \\\\\n",
-    "  \\frac{\\partial u_y}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla u_y = - \\frac{1}{\\rho} \\nabla p + \\eta d\n",
+    "  \\frac{\\partial u_y}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla u_y &= - \\frac{1}{\\rho} \\nabla p + \\eta d\n",
     "  \\\\\n",
-    "  \\text{subject to} \\quad \\nabla \\cdot \\mathbf{u} = 0,\n",
+    "  \\text{s.t.} \\quad \\nabla \\cdot \\mathbf{u} &= 0,\n",
     "  \\\\\n",
-    "  \\frac{\\partial d}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla d = 0 \n",
+    "  \\frac{\\partial d}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla d &= 0 \n",
-    "$\n",
+    "\\end{aligned}$\n",
     "\n",
     "As optimization objective we'll consider a more difficult variant of the previous example: the state of the observed density $d$ should match a given target after $n=20$ steps of simulation. In contrast to before, the marker $d$ cannot be modified in any way, but only the initial state of the velocity $\\mathbf{u}$ at $t=0$. This gives us a split between observable quantities for the loss formulation, and quantities that we can interact with during the optimization (or later on via NNs).\n",
     "\n",

intro-teaser.ipynb

@@ -0,0 +1,418 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# A Teaser Example"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's directly look at a very reduced example that highlights some of the key capabilities of physics-based learning approaches.\n",
+    "\n",
+    "Take a look at the following picture - the desired solution is shown in light gray. If we don't take care we'll learn approximations like the red one shown in the left, which are completely off! With an improved learning setup, ideally by using a discretized numerical solver, we can at least accurately represent a part of the solutions (green on the right).\n",
+    "\n",
+    "```{figure} resources/intro-teaser-side-by-side.png\n",
+    "---\n",
+    "height: 240px\n",
+    "name: intro-teaser-side-by-side\n",
+    "---\n",
+    "Side by side - supervised versus differentiable physics training.\n",
+    "```\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Differentiable physics"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's illustrate the properties of deep learning via differentiable physics (DP) with a simple example: We'd like to find an unknown function $f^*$ that generates solutions from a space $Y$ that we're interested in. \n",
+    "\n",
+    "Let's additionally assume we have a generic differential equation $\\mathcal P^*: Y \\to X$ (our _model_ equation), that encodes a property of the solutions, here modeled in terms of a mapping to the input space $X$. Alternatively, we could imagine $X$ representing future solutions that we're looking for (then $\\mathcal P^*$ would model a time evolution), or it could be a constraint for conservation of mass (then $\\mathcal P^*$ would measure divergence).\n",
+    "\n",
+    "Using a neural network $f$ to learn the unknown and ideal function $f^*$, we could turn to classic _supervised_ training to obtain $f$ by collecting data. This classical setup requires a dataset by sampling $x$ from $X$ and adding the corresponding solutions $y$ from $Y$. We could obtain these, e.g., by classical numerical techniques. Then we train the NN $f$ in the usual way using this dataset. \n",
+    "\n",
+    "In contrast to this, differentiable physics takes advantage of the fact that we can directly use a discretized version of the physical model $\\mathcal P$ and employ it to guide the training of $f$. I.e., we want $f$ to _interact_ with our _simulator_ $\\mathcal P$. This can vastly improve the learning, as we'll illustrate below with a super simple example, and later on with more complex ones.\n",
+    "\n",
+    "(Note that it order for the DP approach to work, $\\mathcal P$ has to differentiable, as implied by the name. These differentials, in the form of a gradient, are what's driving the learning process.)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Finding the inverse function of a parabola"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To illustrate these two approaches, we consider the following simplified setting: Given the function $\\mathcal P: y\\to y^2$ for $y$ in the inverval $[0,1]$, find the unknown function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$. Note that the _discretization_ here is simply given by representing the $x$ and $y$ via floating point numbers in the computer.\n",
+    "\n",
+    "We know possible solutions for $f$ are the positive or negative square root function (for completeness: piecewise combinations would also be possible).\n",
+    "\n",
+    "We can try to solve this by using a neural network to approximate the inverse mapping $f$.\n",
+    "Let's start with the supervised approach."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import tensorflow as tf\n",
+    "import matplotlib.pyplot as plt"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For supervised training, we need a method to find for each datapoint the corresponding solution of the above problem. We simply use our solver $\\mathcal P$ for the problem to pre-compute these solutions: We randomly choose between the positive and the negative square root. This makes sense because in the generic case, when we require optimization techniques to do this step, these methods are not expected to favor one particular mode in multimodal solutions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# X-Data\n",
+    "N = 200\n",
+    "X = np.random.random(N)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Generation Y-Data\n",
+    "sign = (- np.ones((N,)))**np.random.randint(2,size=N)\n",
+    "Y = np.sqrt(X) * sign"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we can define a network, loss, and training configuration. We'll use a simple `keras` model with three hidden layers, ReLU activations."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Neural network\n",
+    "act = tf.keras.layers.ReLU()\n",
+    "model_sv = tf.keras.models.Sequential([\n",
+    "  tf.keras.layers.Dense(8, activation=act),\n",
+    "  tf.keras.layers.Dense(8, activation=act),\n",
+    "  tf.keras.layers.Dense(1,activation='linear')])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Loss function\n",
+    "Loss_sv = tf.keras.losses.MeanSquaredError()\n",
+    "model_sv.compile(optimizer='adam', loss=Loss_sv)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
+      "Instructions for updating:\n",
+      "If using Keras pass *_constraint arguments to layers.\n",
+      "Train on 200 samples\n",
+      "Epoch 1/5\n",
+      "200/200 [==============================] - 1s 3ms/sample - loss: 0.5135\n",
+      "Epoch 2/5\n",
+      "200/200 [==============================] - 0s 355us/sample - loss: 0.5097\n",
+      "Epoch 3/5\n",
+      "200/200 [==============================] - 0s 402us/sample - loss: 0.5077\n",
+      "Epoch 4/5\n",
+      "200/200 [==============================] - 0s 399us/sample - loss: 0.5066\n",
+      "Epoch 5/5\n",
+      "200/200 [==============================] - 0s 303us/sample - loss: 0.5054\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Training\n",
+    "results_sv = model_sv.fit(X, Y, epochs=5, batch_size= 5,verbose=1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "As both model and data set are very small, the training converges very quickly, but if we inspect the predictions of the network, we can see that it nowhere near the solution we wer hoping to find: it averages between the data points on both sides of the x-axis and therefore, fails to find satisfying solutions to our above problem.\n",
+    "\n",
+    "The following plots nicely highlights this: it shows the data in blue, and the supervised solution in red. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de3ycdZ33/9en06ZpetgmpLQNpScp0AqlYEC6KJZVBJWbooCWc5fTIltd11WsD9ZbRPnZ1VXum3tZpSoCupxWEKquC1iIsFKEFKpAEVrSAj2SNgk9pKVt+vn9cV1Xes2Va5JJmmSSzPv5eMwjc52/18xkPvM9m7sjIiLSWYMKnQAREemfFEBERKRLFEBERKRLFEBERKRLFEBERKRLFEBERKRLFEBE2mFmc8xsXTeeb7KZuZkN7q5z9jVmttbMPlLodEjPUwCRPs/MPmBmT5vZO2bWYGZ/MLMTw23zzex/Cp1GkWI0YH8FycBgZqOAXwOfBe4HSoAPAu8WMl35MLPB7r6v0OlI05fTJv2HciDS1x0J4O73uHuLu+9y90fd/c9mNh34ITDbzHaYWROAmX3CzF4ws21m9paZ3RCdLFaEdJmZvWlmW8zs+tj2YWZ2h5k1mtlK4MR4YsxsoZm9bmbbzWylmX0ytm1+mDu62cy2AjeYWcbM/jW8Th3wifZuNs/z/1uYG/uLmX04tr3GzL5tZs+G9/6wmVUk7vsKM3sTeDxcf7aZvWxmTeHx0/NJS7j9KjN7Jbb9hNjmWWb25zCd95lZaXv3Lf2Uu+uhR599AKOArcCdwMeA8sT2+cD/JNbNAY4l+IE0E9gMnBNumww48CNgGHAcQW5merh9EfAUUAEcDrwErIud+3ygKjz3Z4CdwPhYWvYBnyPI3Q8DrgH+Ep6rAngivP7gHPebz/n/ERgSbn8HqAi31wDrgWOA4cADwM8T931XuG0YQXDeCZwenu86YDVQkkdazg+vdSJgwBHApHDbWuDZ8NgK4BXgmkJ/lvTogf/PQidADz06egDTgTuAdeEX6BJgbLitTQBJOf7/ADeHz6Mv0gmx7c8C88LndcCZsW1XxwNIyrlXAHNjaXkzsf3x+Jcn8NH2Akge598AWCLtl4TPa4BFsW0zgD1AJnbfU2PbvwbcH1seFAaFOXmk5RHgH3Lstxa4OLb8HeCHhf4c6dH9DxVhSZ/n7q+4+3x3n0Dw67qKICikMrP3m9kTZlZvZu8Q5AIqE7ttij1vBkaEz6uAt2Lb3kic+1IzWxEW+TSF6YmfO35sh+dLSXtH51/v4bdy7HxVOa7/BkHOIlf6quLpcff94fbD8kjL4cDr7dxKrtdXBhAFEOlX3P0vBLmRY6JVKbvdTZBLOdzd/4qgnsTyvMRGgi/HyMToiZlNIij6WgAc4u6jCYq44udOpifn+ZLyPP9hZhZfnkiQK4kkr7UX2JIjfRuASbHrW3j8+jzS8hbwnlz3IsVBAUT6NDM72sz+ycwmhMuHAxcAz4S7bAYmmFlJ7LCRQIO77zazk4ALO3HJ+4Gvmll5eM3PxbYNJ/gCrg/T8rccCGTtne/zZjbBzMqBhe3sm8/5Dw3PN8TMzico3vuv2PaLzWyGmZUBNwK/cPeWdtL2CTP7sJkNAf6JoD7o6TzS8mPgS2b2PgscEQYdKSIKINLXbQfeD/zRzHYSBI6XCL7sIKhjeBnYZGbRL+1rgRvNbDvwvwm+KPP1DYJinTXAo8DPog3uvhL4HrCMIHAdC/yhg/P9iKC+4E/A88CDuXbM8/x/BKYR5CpuAs5z962x7T8jyKFtAkqBz7dzvVeBi4H/F57vfwH/y933dJQWd//P8Pp3E7xHDxFUmEsRseziVBHpq8xsPnClu38gx/YaglZXP+7NdEnxUg5ERES6RAFERES6REVYIiLSJcqBiIhIlxTVYIqVlZU+efLkQidDRKRfWb58+RZ3H5NcX1QBZPLkydTW1hY6GSIi/YqZpY6goCIsERHpEgUQERHpEgUQERHpkqKqA0mzd+9e1q1bx+7duwudlKJQWlrKhAkTGDJkSKGTIiIHqaABxMxuB84C3nb3NoPShaOD/l/g4wRDQs939+fDbZcB/xzu+i13v7MraVi3bh0jR45k8uTJZA9yKt3N3dm6dSvr1q1jypQphU6OiBykQhdh3QGc2c72jxEMHDeNYGKfHwCE03R+nWCQvZOAr4cjnXba7t27OeSQQxQ8eoGZccghhyi3JzJAFDSAuPuTQEM7u8wF7vLAM8BoMxsPnAE85u4N7t4IPEb7gahdCh69R6+1SO9qaGhg7dq1NDS091XbNX29DuQwsmdQWxeuy7VeRKQoNTc3s3PnToYPH05ZWRkQBI8NG4L5xnbs2AFARUX3jbpf6CKsHmdmV5tZrZnV1tfXFzo5qTKZDLNmzeK9730vxx13HN/73vfYv39/u8esXbuWu+++u8fTduWVV7Jy5cp293nooYc63EdEukdzczP19fU0Nze3Pm9oaGDNmjVs3ryZNWvW0NzcDMC2bduyjk0uH6y+ngNZT/YUnRPCdeuBOYn1NWkncPfFwGKA6urqPjly5LBhw1ixYgUAb7/9NhdeeCHbtm3jG9/4Rs5jogBy4YWdmWyv8378446nlnjooYc466yzmDFjRo+mRaQYxHMSQFauorm5mTVr1hANgmtmJAfEdXd27txJWVkZo0aNas15AIwaNapb09rXcyBLgEvDKTNPBt5x940EM7x9NJx2tBz4aLiuV8R/AXS3Qw89lMWLF/Nv//ZvuDtr167lgx/8ICeccAInnHACTz/9NAALFy7kqaeeYtasWdx8880594tbu3YtRx99NBdddBHTp0/nvPPOa72HpUuXcvzxx3Psscdy+eWX8+677wIwZ86c1uFfRowYwfXXX89xxx3HySefzObNm3n66adZsmQJX/7yl5k1axavv/46t9xyCzNmzGDmzJnMmzev218jkYEg+T3S3NzMG2+8QV1dHZs3b6aurq5NrmLnzp1ZASNtNHUzaw0+FRUVVFVVMWLECKqqqrq1+AoK34z3HoKcRKWZrSNoWTUEwN1/SDDX88eB1QTNeP823NZgZt8EngtPdaO7d38NUYr4LwAzY8qUKa3ljd1l6tSptLS08Pbbb3PooYfy2GOPUVpayqpVq7jggguora1l0aJF/Ou//iu//vWvW9OVtl/Sq6++yk9+8hNOOeUULr/8cv793/+dBQsWMH/+fJYuXcqRRx7JpZdeyg9+8AO+8IUvZB27c+dOTj75ZG666Sauu+46fvSjH/HP//zPnH322Zx11lmcd955ACxatIg1a9YwdOhQmpqauvW1ERkINm3axJYtwQzMZsb48ePZuHFjam4i+hvlROK5juh5dI6WlpasOhAIgkh3B45IQQOIu1/QwXYH/j7HttuB23siXe2J/wKIZxV7yt69e1mwYAErVqwgk8nw2muvHdR+hx9+OKeccgoAF198Mbfccgunn346U6ZM4cgjjwTgsssu49Zbb20TQEpKSjjrrLMAeN/73sdjjz2Weo2ZM2dy0UUXcc4553DOOed06b5F+qu0yuy4hoaG1uABwffItm3bcuYmogARnW/KlCk5i7h6W1+vA+lz4r8A4lnF7lRXV0cmk+HQQw/lG9/4BmPHjuVPf/oT+/fvp7S0NPWYm2++Oa/9ks1oO9OsdsiQIa37ZzIZ9u3bl7rfb37zG5588kl+9atfcdNNN/Hiiy8yeLA+ajIwJessOiqhSKvIHjVqVJviqaqqKkpLS9sEiLKysqxzFiJwRPRf3UnJXwDd/ebV19dzzTXXsGDBAsyMd955hwkTJjBo0CDuvPNOWlpaABg5ciTbt29vPS7Xfklvvvkmy5YtY/bs2dx999184AMf4KijjmLt2rWsXr2aI444gp/97Gd86EMfyjvN8bTs37+ft956i9NOO40PfOAD3HvvvezYsYPRo0cfxKsiUjhRgGhpaWH37t2MGjWqtUgoWaQ9evToDksokhXblZWVVFRUUFpaSmNjIwDl5eVZAaOvUgDpguQvgIO1a9cuZs2axd69exk8eDCXXHIJX/ziFwG49tprOffcc7nrrrs488wzW3/lzJw5k0wmw3HHHcf8+fNz7pd01FFHceutt3L55ZczY8YMPvvZz1JaWspPf/pTzj//fPbt28eJJ57INddck3f6582bx1VXXcUtt9zCvffeyxVXXME777yDu/P5z39ewUP6lShgZDIZdu3aRVNTU1bOIN6fIlmkDW2LnZKi4LNt27asYNTd3yu9oajmRK+urvZkxfIrr7zC9OnTC5Si3rV27VrOOussXnrppYKmo5hec+nbGhoasr7Ik81kcxkxYgSTJ09ObVQDha2X6Almttzdq5PrlQMRkQEv317aLS0tHQYPONCfIleR9kAJHB1RACkikydPLnjuQ6QnxYufoiatkF6xndZL+9BDD03tnFdeXk4mk2lTBwL9s+ipuyiAiMiAkFb81F7Fdlov7XiOIh6EijVAdEQBRET6vObm5tQWSnHJZrDQfsX2QKrMLhQFEBEpuGRfiniwALJyFo2NjUydOrXNl3yylzYEgaO8vJzy8vLUiu2e7KVdDBRARKSgki2Z4gGgsbGR8vLyNjmLtP4VHRU/KVfR/fr6YIpF4aabbuK9730vM2fOZNasWfzxj38sSDr++q//+qDPUVNT0zrciUhcrkFI0/pSJCVHTMjVz6msrIwxY8ZQUVHBmDFjFDR6mHIgBbZs2TJ+/etf8/zzzzN06FC2bNnCnj17euRa7o67M2hQ+u+GtBF8RTorrclse4OQJocHSmsBVV5e3mEdiPQ+5UC6Ytky+Pa3g78HaePGjVRWVjJ06FAgGNagqqqKyZMntw64Vltby5w5cwC44YYbuOSSS5g9ezbTpk3jRz/6Ueu5vvvd73LiiScyc+ZMvv71rwNB58GjjjqKSy+9lGOLine truncated
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "# Results\n",
+    "plt.plot(X,Y,'.',label='Data points', color=\"lightgray\")\n",
+    "plt.plot(X,model_sv.predict(X),'.',label='Supervised', color=\"red\")\n",
+    "plt.xlabel('y')\n",
+    "plt.ylabel('x')\n",
+    "plt.title('Standard approach')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "😱 This is obviously completely wrong! The red solution is nowhere near one of the two modes of our solution shown in blue.\n",
+    "\n",
+    "Note that the line is often not perfectly at zero, which is where the two modes of the solution should average out in the continuous setting. This is caused by the relatively coarse sampling with only 200 points in this example.\n",
+    "<br>\n",
+    "\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Finding f via a Differentiable Physics approach"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's apply the differentiable physics approach: we'll directly include our discretized model $\\mathcal P$ in the training. \n",
+    "\n",
+    "There is no real data generation step; we only need to sample from the $[0,1]$ interval. We'll simply keep the same $x$ locations used in the previous case."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# X-Data\n",
+    "# X = X , we can directly re-use the X from above, nothing has changed..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Model\n",
+    "model_dp = tf.keras.models.Sequential([\n",
+    "  tf.keras.layers.Dense(8, activation=act),\n",
+    "  tf.keras.layers.Dense(8, activation=act),\n",
+    "  tf.keras.layers.Dense(1,activation='linear')])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This loss function is the **crucial** point now: we directly incorporate the function f into the loss. In this simple case, the `Loss_dp` function simply computes the square of the prediction `y_pred`. \n",
+    "\n",
+    "Later on, a lot more could happen here: we could evaluate finite difference stencils on the predicted solution, or compute a whole implicit time-integration step of a solver. Note: here we effectively have a simple residual equation $y_{\\text{pred}}^2 - y_{\\text{true}} = 0$ which we are minimizing via a _mean-squared error_. It's not necessary to make it so simple: the more knowledge and numerical methods we can incorporate, the better we can guide the training process."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#Loss\n",
+    "mse = tf.keras.losses.MeanSquaredError()\n",
+    "def Loss_dp(y_true, y_pred):\n",
+    "    return mse(y_true,y_pred**2)\n",
+    "\n",
+    "model_dp.compile(optimizer='adam', loss=Loss_dp)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
+      "Instructions for updating:\n",
+      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
+      "Train on 200 samples\n",
+      "Epoch 1/5\n",
+      "200/200 [==============================] - 0s 1ms/sample - loss: 0.2711\n",
+      "Epoch 2/5\n",
+      "200/200 [==============================] - 0s 282us/sample - loss: 0.0733\n",
+      "Epoch 3/5\n",
+      "200/200 [==============================] - 0s 321us/sample - loss: 0.0028\n",
+      "Epoch 4/5\n",
+      "200/200 [==============================] - 0s 322us/sample - loss: 0.0013\n",
+      "Epoch 5/5\n",
+      "200/200 [==============================] - 0s 335us/sample - loss: 0.0012\n"
+     ]
+    }
+   ],
+   "source": [
+    "#Training\n",
+    "results_dp = model_dp.fit(X, X, epochs=5, batch_size= 5,verbose=1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now the network actually has learned a good inverse of the parabola function! The following plot shows the solution in green."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deXxU9bn48c+TCSEEiCQhLGFLqNSrgkSNLFppKGiRqtSlrQsCVkttS1tta6W3t7fW9l6pP6v21raKK7TuO1otKhC1GpUgoBZEMATDHkhigAFCJs/vj3NmPDOZyTIkmSzP21dezJzzPWe+38mYZ767qCrGGGNMSyUlOgPGGGM6Jwsgxhhj4mIBxBhjTFwsgBhjjImLBRBjjDFxsQBijDEmLhZATIuIyF0i8ivP8++JyC4R2S8iWSJyhohsdJ9/PZF5dfMXlt8m0haJyNUxzuWKiIpIcuvmEESkTESmtvI9XxKR2a15z85ORB4Ukd8lOh9dSav/z2A6LxEpAwYCdUAAWAcsBhaqaj2Aql7jSd8DuA2YoKpr3WM3AXeq6h/bN/cgInOAq1X1S8Fj3vx2J6p6TqLzYLo+q4GYSOepal9gBLAAuAG4L0bagUAq8G/PsRERz5utLb7dm87FPgOdiwUQE5WqfqaqS4BvAbNFZDR83gwgIl8ENrjJq0VkuYh8AowEnnebsHqKyDEicp+I7BCRbe61Pvdec0TkTRG5XUT2Aje6x78tIutFpEpElorIiGC+3Gaka9xmsmoR+bM4jgfuAia6r13tza/7OENEXhCRCvfeL4jI0Iiif0FE3hWRGhF5TkQyo70/jZUrStobReRJEXlMRPaJyHsiMjYiWb6IvC8in7npUt1rPxSR8zz36iEie0TkZBFJFZG/i8he971YKSID3XRhzXEi8h33Pd0nIutE5BT3+A1u/veJyAYRmRKjDF8TkdXu+1IuIjd6zgWb9+aKyHb3PflZc8vvNuHdICLvAwdEJFlEjnfLUC0i/xaR85uTF/f8l0TkLffacrdmGpQhIv9w8/GOiHwhWnlNM6mq/dgPqgpQBkyNcvxT4Hvu4weB37mPcwEFkmPdA3gGuBvoDQwA3gW+656bg9Nc9kOc5tRewAxgE3C8e+y/gLc891PgBaAfMByoAKZ57veviLx785sFXASkAX2BJ4BnPWmLgG3AaDe/TwF/j1bWxsoV5f27ETgCXAz0AH4GbAZ6eN6zd4EcIBNYD1zjnvs58JjnXjOAD9zH3wWed8vjA04F0j1ludp9/A23XKcBAhyLU1M8DigHcjxl/EKMMhQCY3C+dJ4E7AK+HvHePOK+H2Pc38vUFpR/DTDM/Qz0cD8D/wmkAF8B9gHHNSMvI9y0l7r3yQLyPZ+FvcA4nM/WQ8Cjif7/rjP/WA3ENMd2nD9sLeJ+G54OXKuqB1R1N3A7cIn33qr6J1WtU9WDwDXAzaq6XlXrgP/F+XY+wnPNAlWtVtVPgRVAfnPyo6p7VfUpVfWr6j7gf4AvRyT7m6p+qKoHgF8B34ysWTSzXJFWqeqTqnoEp98oFZjgOf9/qrpdVStxgkKwTH8HpotIuvv8CuBv7uMjOH8gj1XVgKquUtWaKK99NXCLqq5UxyZV3YLTz9UTOEFEeqhqmap+EuO9K1LVD1S1XlXfxwkWke/db9z34wPgAZw/4i0pf7n7GZgA9MH5Pdeq6nKcLw2XNiMvlwGvquojqnrE/Z2v8bzOM6r6rvvZeohmfnZMdBZATHMMASrjuG4EzrfAHW5zQjXOt/YBnjTlUa75oyd9Jc635iGeNDs9j/04f2yaJCJpInK3iGwRkRrgdaBfRIDw5meLm//+cZQrUui+6gxI2IpT42i0TKq6HXgTuEhE+gHn4PzhAyeQLAUedZuObhFnYEOkYUCDwKCqm4BrcWoIu0XkURHJiUwHICLjRWSF2/z3GU6gj3xfIt+7nGjnYpTfe20OUO6m895vSDPyErWsHnF9dkx0FkBMo0TkNJz/cf8Vx+XlwGGgv6r2c3/SVfVET5rI5aDLcZqC+nl+eqnqW814vaaWlv4pTrPNeFVNBya5x8WTZpjn8XCcb/l74ihXpNB9RSQJGIpTs2uORcBMnKaoYlXdBuB+w/6Nqp4AnA6cC8yKcn05ELWtX1UfVmfU2gic9+/3MfLwMLAEGKaqx+D0N0lEmsj3bnu0czHK7/3dbQeGuem899vWjLzELKtpfRZATFQiki4i5wKP4vQDfNDSe6jqDuBl4A/u/ZJE5AsiEtn04XUX8AsROdHNxzEi8o1mvuQuYKiIpMQ43xc4iNPpnwn8OkqamSJygoikATcBT6pqoBXKdaqIXCjOKKNrcQLQ280s17PAKcCPcYZVAyAik0VkjFuDqsEJdvVRrr8X+JmInCqOY0VkhIgcJyJfEZGewCGc9yba9eC8d5WqekhExuE0FUX6lVvLOxG4EngszvK/g1M7+Lk4gwYKgfNwPotN5eUhYKqIfNPtjM8SEWumaiMWQEyk50VkH843uV/itFdfeRT3m4XTEboOqAKeBAbHSqyqz+B8C37UbWb6EKfZpjmW4wwh3ikikbUGgDtwOmn34Pzx+meUNH/D6WzdidNO/6MYr9WicgHP4Yxoq8Lpx7jQ7Q9oktsv8BSQBzztOTXIfd0anI731/i8f8R7/RM4/T0P43QwP4vTp9UTZ6j2HpzyDgB+ESMb3wducj8b/w08HiXNazid38uAW1X1Zc+5ZpdfVWtxAsY5bt7+AsxS1Y+ayovbLzYdp7ZZidM5HznizbQSUbUNpYxpS+4w02NVdeZR3OO/gS8ezT3aiojk8vmoqroo52/kKMtvOiabtGNMB+c2t12F883dmA7DmrCM6cBE5Ds4zYkvqerric6PMV7WhGWMMSYuVgMxxhgTl27VB9K/f3/Nzc1NdDaMMaZTWbVq1R5VzY483q0CSG5uLiUlJYnOhjHGdCoisiXacWvCMsYYExcLIMYYY+JiAcQYY0xculUfSDRHjhxh69atHDp0KNFZ6bJSU1MZOnQoPXpEWyjWGNNZJTSAiMj9OCuI7lbV0VHOC/BHnLVt/MAcVX3PPTcbZ7MhcDYMWhRPHrZu3Urfvn3Jzc3FeTnTmlSVvXv3snXrVvLy8hKdHWNMK0p0E9aDwLRGzp8DjHJ/5gJ/hdDSDr8GxuPsLvZrEcmIJwOHDh0iKyvLgkcbERGysrKshmdMF5TQAOIuzdDYRkUzgMXuLmpv42z+Mxj4KvCKqlaqahXwCo0HokZZ8Ghb9v4akziVlZWUlZVRWRnPnnCN6+h9IEMI36lsq3ss1nFjjOmW/H4/Bw4coHfv3qSlpQFO8Ni+3dm3a//+/QBkZrZ4d+qYEt2E1eZEZK6IlIhISUVFRaKzE5XP5yM/P58TTzyRsWPH8oc//IH6+lj7+jjKysp4+OGH2yQ/06dPp7q6utE0Dz74YOiDaYxpP36/n4qKCvx+f+hxZWUlmzdvZteuXWzevBm/3w9ATU1N2LWRz49WR6+BbCN8m8yh7rFtQGHE8aJoN1DVhcBCgIKCgg65cmSvXr1Ys2YNALt37+ayyy6jpqaG3/zmNzGvCQaQyy6LtjHc0XnxxRebTPPggw8yevRocnKibqFtjImTtyYBhNUq/H4/mzdvJrgIrogQuSCuqnLgwAHS0tJIT08P1TwA0tPTWzWvHb0GsgSY5W7DOQH4zN1OdClwtohkuJ3nZ7vH2oX3G0BrGzBgAAsXLuTOO+9EVSkrK+PMM8/klFNO4ZRTTuGtt5ytwefPn88bb7xBfn4+t99+e8x0RUVFTJo0ia997Wscd9xxXHPNNaHazSOPPMKYMWMYPXo0N9xwQygPubm57Nmzh7KyMo4//ni+853vcOKJJ3L22Wdz8OBBnnzySUpKSrj88svJz8/n4MGDzJ8/nxNOOIGTTjqJn/3sZ63+vhjT1UT+HfH7/WzZsoXS0lJ27dpFaWlpg1rFgQMHwgJGtNXURSQUfDIzM8nJyaFPnz7k5OS0avMVJH4Y7yM4NYn+IrIVZ2RVDwBVvQt4EWcI7yacYbxXuucqReS3wEr3Vjepauv3EEXh/QYgIuTl5YXaG1vLyJEjCQQC7N69mwEDBvDKK6+QmprKxo0bufTSSykpKWHBggXceuutvPDCC6F8RUsH8O6777Ju3TpGjBjBtGnTePrppzn99NO54YYbWLVqFRkZGZx99tk8++yzfP3rXw/Ly8aNG3nkkUe45557+OY3v8lTTz3FzJkzufPOO7n11lspKChg7969PPPMM3z00UeISJPNX8Z0dzt37mTPHmfXZRFh8ODB7NixI2ptIvhvsCbirXUEHwfvEQgEwvpAwAkirR04ghIaQFT10ibOK/CDGOfuB+5vi3w1xvsNwFtVbCtHjhxh3rx5rFmzBp/Px8cff9zidOPGjWPkyJEAXHrppfzrX/+iR48eFBYWkp3tLLB5+eWX8/rrrzcIIHl5eeTn5wNw6qmnUlZW1uC1jznmGFJTU7nqqqs499xzOffcc1uj6MZ0StE6s70qKytDwQOcvyM1NTUxaxPBABG8X15eXswmrvbW0ftAOhzvNwBvVbE1lZaW4vP5GDBgAL/5zW8YOHAga9eupb6+ntTU1KjX3H777THTRQ6jbcmw2p49e4Ye+3w+Dh482CBNcnIy7777LsuWLePJJ5/kzjvvZPny5c1+DWM6s8g+i6ZaKKJ1ZKenpzdonsrJySE1NbVBgEhLSwu7ZyICR5AFkBaK/AbQ2r+8iooKrrnmGubNm4eI8NlnnzF06FCSkpJYtGgRgUAAgL59+7Jv377QdbHSgdOEtXnzZkaMGMFjjz3G3LlzGTduHD/60Y/Ys2cPGRkZPPLII/zwhz9sdj69r79//378fj/Tp0/njDPOCNV2jOnMgoHB5/MRCAQIBAIcOnSI9PT0UJNQZJN2v379mmyhiOzY7t+/P5mZmaSmplJLine truncated
+      "text/plain": [
+       "<Figure size 432x288 with 1 Axes>"
+      ]
+     },
+     "metadata": {
+      "needs_background": "light"
+     },
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "# Results\n",
+    "plt.plot(X,Y,'.',label='Datapoints', color=\"lightgray\")\n",
+    "#plt.plot(X,model_sv.predict(X),'.',label='Supervised', color=\"red\") # optional for comparison\n",
+    "plt.plot(X,model_dp.predict(X),'.',label='Diff. Phys.', color=\"green\") \n",
+    "plt.xlabel('x')\n",
+    "plt.ylabel('y')\n",
+    "plt.title('Differentiable physics approach')\n",
+    "plt.legend()\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This looks much better 😎, at least in the range of 0.2 to 1. \n",
+    "\n",
+    "What has happened here?\n",
+    "\n",
+    "- We've prevented an undesired averaging of multiple modes in the solution by evaluating our discrete model w.r.t. current prediction of the network, rather than using a pre-computed solution. This let's us find the best single mode near the network prediction, and prevents an averaging of the modes that are contained in the solution manifold.\n",
+    "\n",
+    "- We're still only getting one side of the curve! This is to be expected, because we're representing the solutions with a deterministic function. Hence we can only represent a single mode. Interestingly, whether it's the top or bottom mode is determined by the random initialization of the weights in $f$ - run the example a couple of time to see this effect in action. To capture multiple modes we'd need to extend the model to capture the full distribution of the outputs and parametrize it with additional dimensions.\n",
+    "\n",
+    "- The region with $x$ near zero is typically still off in this example. The is caused by the larger curvature in this region, and the even sampling with points along $x$. This could be alleviated with more points, a larger network, and potentially longer training.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Discussion\n",
+    "\n",
+    "It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and often it's not clear where (and how many) exist in the solution manifold we're interested in. Then supervised learning is very dangerous - we might simply and unknowingly _blur_ out these different modes.\n",
+    "\n",
+    "A good examples are bifurcations in fluid flows - the smoke rising above a candle will start out straight, and then, due to tiny perturbations in its motion, start oscillating in a random direction. The images below illustrate this case via _numerical perturbations_: the perfectly symmetric setup will start turning left or right, depending on how the approximation errors build up. Similarly, we'll have different modes in all our numerical solutions, and typically it's important to recover them, rather than averaging out. Hence, we'll show how to leverage training via _differentiable physics_ in the following chapters for more practical and complex cases.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Next steps\n",
+    "\n",
+    "- The example above is itentionally using a fairly \"bad\" setup. Try to find out how to obtain a solution such as the green one shown in the very first image at the top.\n",
+    "\n",
+    "- Or you can try extending the setup to a 2D case, i.e. a paraboloid. Given the function $\\mathcal P:(y_1,y_2)\\to y_1^2+y_2^2$, find an inverse function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

intro.md

@@ -15,11 +15,12 @@ more tightly coupled learning algorithms with differentiable simulations.
 height: 220px
 name: pbdl-teaser
 ---
-Some examples ... preview teaser ...
+Some visual examples of hybrid solvers, i.e. numerical simulators that are enhanced by trained neural networks.
 ```
 % Teaser, simple version:
 % ![Teaser, simple version](resources/teaser.png)
 
+## Coming up
 
 As a _sneak preview_, in the next chapters we'll show:
 
@@ -37,10 +38,7 @@ If you find mistakes, please also let us know! We're aware that this document is
 and we're eager to improve it. Thanks in advance!
 
 This collection of materials is a living document, and will grow and change over time. 
-Feel free to contribute 😀
+Feel free to contribute 😀 
-
-[TUM Physics-based Simulation Group](https://ge.in.tum.de).
-
 We also maintain a [link collection](https://github.com/thunil/Physics-Based-Deep-Learning) with recent research papers.
 
 ```{admonition} Code, executable, right here, right now
@@ -52,16 +50,6 @@ immediately see what happens -- give it a try...
 Oh, and it's great because it's [literate programming](https://en.wikipedia.org/wiki/Literate_programming).
 ```
 
-## More Specifically
-
-To be a bit more specific, _physics_ is a huge field, and we can't cover everything... 
-
-```{note}
-For now our focus is:
-- field-based simulations , less Lagrangian
-- simulations, not experiments
-- combination with _deep learning_ (plenty of other interesting ML techniques)
-```
 
 ---
 

overview-equations.md

@@ -1,10 +1,12 @@
 Model Equations
 ============================
 
-overview of PDE models to be used later on ...
+TODO
+
+give an overview of PDE models to be used later on ...
 continuous pde $\mathcal P^*$
 
-$\vx \in \Omega \subseteq \mathbb{R}^d$ 
+$\mathbf{x} \in \Omega \subseteq \mathbb{R}^d$ 
 for the domain $\Omega$ in $d$ dimensions,
  time $t \in \mathbb{R}^{+}$.
 
@@ -47,11 +49,8 @@ and the abbreviations used inn: {doc}`notation`, at the bottom of the left panel
 % \newcommand{\corr}{\mathcal{C}}                         % just C for now...
 % \newcommand{\nnfunc}{F} % {\text{NN}}
 
-
 Some notation from SoL, move with parts from overview into "appendix"?
 
-
-
 We typically solve a discretized PDE $\mathcal{P}$ by performing discrete time steps of size $\Delta t$. 
 Each subsequent step can depend on any number of previous steps,
 $\mathbf{u}(\mathbf{x},t+\Delta t) = \mathcal{P}(\mathbf{u}(\mathbf{x},t), \mathbf{u}(\mathbf{x},t-\Delta t),...)$, 
@@ -83,12 +82,13 @@ $\mathbf{u} = (u_x,u_y,u_z)^T$ for $d=3$.
 
 Burgers' equation in 2D. It represents a well-studied advection-diffusion PDE:
 
-$\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x =
+$\begin{aligned}
+  \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &=
   \nu \nabla\cdot \nabla u_x + g_x(t), 
   \\
-  \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y =
+  \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &=
   \nu \nabla\cdot \nabla u_y + g_y(t)
-$, 
+\end{aligned}$, 
 
 where $\nu$ and $\mathbf{g}$ denote diffusion constant and external forces, respectively.
 
@@ -104,15 +104,15 @@ Later on, additional equations...
 
 Navier-Stokes, in 2D:
 
-$
+$\begin{aligned}
-    \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x =
+    \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &=
     - \frac{1}{\rho}\nabla{p} + \nu \nabla\cdot \nabla u_x  
     \\
-    \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y =
+    \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &=
     - \frac{1}{\rho}\nabla{p} + \nu \nabla\cdot \nabla u_y  
     \\
-    \text{subject to} \quad \nabla \cdot \mathbf{u} = 0
+    \text{subject to} \quad \nabla \cdot \mathbf{u} &= 0
-$
+\end{aligned}$
 
 
 
@@ -121,28 +121,29 @@ Navier-Stokes, in 2D with Boussinesq:
 %$\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x$
 %$ -\frac{1}{\rho} \nabla p $
 
-$
+$\begin{aligned}
-  \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x = - \frac{1}{\rho} \nabla p 
+  \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &= - \frac{1}{\rho} \nabla p 
   \\
-  \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y = - \frac{1}{\rho} \nabla p + \eta d
+  \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &= - \frac{1}{\rho} \nabla p + \eta d
   \\
-  \text{subject to} \quad \nabla \cdot \mathbf{u} = 0,
+  \text{subject to} \quad \nabla \cdot \mathbf{u} &= 0,
   \\
-  \frac{\partial d}{\partial{t}} + \mathbf{u} \cdot \nabla d = 0 
+  \frac{\partial d}{\partial{t}} + \mathbf{u} \cdot \nabla d &= 0 
-$
+\end{aligned}$
-
 
 
 Navier-Stokes, in 3D:
 
 $
-  \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x = - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_x 
+\begin{aligned}
+  \frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &= - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_x 
   \\
-  \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y = - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_y 
+  \frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &= - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_y 
   \\
-  \frac{\partial u_z}{\partial{t}} + \mathbf{u} \cdot \nabla u_z = - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_z 
+  \frac{\partial u_z}{\partial{t}} + \mathbf{u} \cdot \nabla u_z &= - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_z 
   \\
-  \text{subject to} \quad \nabla \cdot \mathbf{u} = 0.
+  \text{subject to} \quad \nabla \cdot \mathbf{u} &= 0.
+\end{aligned}
 $
 
 

overview.md

@@ -11,6 +11,8 @@ a starting point for new researchers as well as a hands-on introduction into
 state-of-the-art resarch topics. 
 
 
+
+
 ```{figure} resources/overview-pano.jpg
 ---
 height: 240px
@@ -127,6 +129,21 @@ starting points with code examples, and illustrate pros and cons of the
 different approaches. In particular, it's important to know in which scenarios 
 each of the different techniques is particularly useful.
 
+
+## More Specifically
+
+To be a bit more specific, _physics_ is a huge field, and we can't cover everything... 
+
+```{note}
+For now our focus are:
+- _field-based simulations_ (no Lagrangian methods)
+- combinations with _deep learning_ (plenty of other interesting ML techniques, but not here)
+- experiments as _outlook_ (replace synthetic data with real)
+```
+
+It's also worth noting that we're starting to build the methods from some very
+fundamental steps. Here are some considerations for skipping ahead to the later chapters.
+
 ```{admonition} You can skip ahead if...
 :class: tip
 
@@ -138,6 +155,9 @@ A brief look at our _Notation_ won't hurt in both cases, though!
 ```
 
 ---
+<br>
+<br>
+<br>
 
 <!-- ## A brief history of PBDL in the context of Fluids
 
@@ -154,6 +174,8 @@ PINNs ... and more ... -->
 
 ## Deep Learning and Neural Networks
 
+TODO
+
 Very brief intro, basic equations... approximate $f^*(x)=y$ with NN $f(x;\theta)$ ...
 
 learn via GD, $\partial f / \partial \theta$ 

physicalloss-code.ipynb

@@ -18,7 +18,7 @@
     "\n",
     "## Preliminaries\n",
     "\n",
-    "Let's just load TF for..."
+    "Let's just load TF and phiflow for now, and initialize the random sampling."
    ]
   },
   {
@@ -54,7 +54,7 @@
     "from phi.tf.flow import *\n",
     "import numpy as np\n",
     "\n",
-    "#rnd = TF_BACKEND  # sample different points in the domain each iteration\n",
+    "#rnd = TF_BACKEND  # for phiflow: sample different points in the domain each iteration\n",
     "rnd = math.choose_backend(1)  # use same random points for all iterations"
    ]
   },