daviddoji/pbdl-book
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fixing typos

Browse Source
This commit is contained in:
NT 2021-06-22 09:49:57 +02:00
parent fbf410fa87
commit 810df88968
8 changed files with 17 additions and 1206 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

656
diffphys-code-ns-v1.ipynb
View File

File diff suppressed because one or more lines are too long

533
diffphys-code-ns-v2a.ipynb
View File

File diff suppressed because one or more lines are too long

12
intro-teaser.ipynb
View File

@ -65,7 +65,7 @@
"id": "latest-amino", "id": "latest-amino",
"metadata": {}, "metadata": {},
"source": [ "source": [
"To illustrate these two approaches, we consider the following simplified setting: Given the function $\\mathcal P: y\\to y^2$ for $y$ in the inverval $[0,1]$, find the unknown function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$. Note: to make things a bit more interesting, we're using $y^2$ here instead of the more common $x^2$ parabola, and the _discretization_ is simply given by representing the $x$ and $y$ via floating point numbers in the computer for this simple case.\n", "To illustrate these two approaches, we consider the following simplified setting: Given the function $\\mathcal P: y\\to y^2$ for $y$ in the intverval $[0,1]$, find the unknown function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$. Note: to make things a bit more interesting, we're using $y^2$ here instead of the more common $x^2$ parabola, and the _discretization_ is simply given by representing the $x$ and $y$ via floating point numbers in the computer for this simple case.\n",
"\n", "\n",
"We know possible solutions for $f$ are the positive or negative square root function (for completeness: piecewise combinations would also be possible).\n", "We know possible solutions for $f$ are the positive or negative square root function (for completeness: piecewise combinations would also be possible).\n",
"We can try to solve this by using a neural network to approximate the inverse mapping $f$.\n", "We can try to solve this by using a neural network to approximate the inverse mapping $f$.\n",
@ -184,9 +184,9 @@
"id": "governmental-mixture", "id": "governmental-mixture",
"metadata": {}, "metadata": {},
"source": [ "source": [
"As both model and data set are very small, the training converges very quickly, but if we inspect the predictions of the network, we can see that it nowhere near the solution we were hoping to find: it averages between the data points on both sides of the x-axis and therefore, fails to find satisfying solutions to our above problem.\n", "As both model and data set are very small, the training converges very quickly, but if we inspect the predictions of the network, we can see that it is nowhere near the solution we were hoping to find: it averages between the data points on both sides of the x-axis and therefore, fails to find satisfying solutions to our above problem.\n",
"\n", "\n",
"The following plots nicely highlights this: it shows the data in blue, and the supervised solution in red. " "The following plots nicely highlights this: it shows the data in light gray, and the supervised solution in red. "
] ]
}, },
{ {
@ -373,7 +373,7 @@
"\n", "\n",
"- We're still only getting one side of the curve! This is to be expected, because we're representing the solutions with a deterministic function. Hence we can only represent a single mode. Interestingly, whether it's the top or bottom mode is determined by the random initialization of the weights in $f$ - run the example a couple of time to see this effect in action. To capture multiple modes we'd need to extend the model to capture the full distribution of the outputs and parametrize it with additional dimensions.\n", "- We're still only getting one side of the curve! This is to be expected, because we're representing the solutions with a deterministic function. Hence we can only represent a single mode. Interestingly, whether it's the top or bottom mode is determined by the random initialization of the weights in $f$ - run the example a couple of time to see this effect in action. To capture multiple modes we'd need to extend the model to capture the full distribution of the outputs and parametrize it with additional dimensions.\n",
"\n", "\n",
"- The region with $x$ near zero is typically still off in this example. The model primarily learns a linear approximation of one half of the parabola here. This is primarily caused by the weak neural network: it is very small and shallow. (Give it a try - it's very easy to fix this in the `model_dp` defition.)\n" "- The region with $x$ near zero is typically still off in this example. The model primarily learns a linear approximation of one half of the parabola here. This is primarily caused by the weak neural network: it is very small and shallow. (Give it a try - it's very easy to fix this in the `model_dp` definition.)\n"
] ]
}, },
{ {
@ -385,7 +385,7 @@
"\n", "\n",
"It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and it's often not clear where (and how many) exist in the solution manifold we're interested in. Using supervised learning is very dangerous in such cases - we might simply and unknowingly _blur_ out these different modes.\n", "It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and it's often not clear where (and how many) exist in the solution manifold we're interested in. Using supervised learning is very dangerous in such cases - we might simply and unknowingly _blur_ out these different modes.\n",
"\n", "\n",
"A good and obvious example are bifurcations in fluid flows - the smoke rising above a candle will start out straight, and then, due to tiny perturbations in its motion, start oscillating in a random direction. The images below illustrate this case via _numerical perturbations_: the perfectly symmetric setup will start turning left or right, depending on how the approximation errors build up. Similarly, we'll have different modes in all our numerical solutions, and typically it's important to recover them, rather than averaging them out. Hence, we'll show how to leverage training via _differentiable physics_ in the following chapters for more practical and complex cases.\n", "Good and obvious example are bifurcations in fluid flows - the smoke rising above a candle will start out straight, and then, due to tiny perturbations in its motion, start oscillating in a random direction. The images below illustrate this case via _numerical perturbations_: the perfectly symmetric setup will start turning left or right, depending on how the approximation errors build up. Similarly, we'll have different modes in all our numerical solutions, and typically it's important to recover them, rather than averaging them out. Hence, we'll show how to leverage training via _differentiable physics_ in the following chapters for more practical and complex cases.\n",
"\n", "\n",
"```{figure} resources/intro-fluid-bifurcation.jpg\n", "```{figure} resources/intro-fluid-bifurcation.jpg\n",
"---\n", "---\n",
@ -407,7 +407,7 @@
"\n", "\n",
"For the simple DP example above:\n", "For the simple DP example above:\n",
"\n", "\n",
"- This notebook is itentionally using a very simple setup. Change the training setup and NN above to obtain a higher-quality solution such as the green one shown in the very first image at the top. \n", "- This notebook is intentionally using a very simple setup. Change the training setup and NN above to obtain a higher-quality solution such as the green one shown in the very first image at the top. \n",
"\n", "\n",
"- Or try extending the setup to a 2D case, i.e. a paraboloid. Given the function $\\mathcal P:(y_1,y_2)\\to y_1^2+y_2^2$, find an inverse function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$.\n", "- Or try extending the setup to a 2D case, i.e. a paraboloid. Given the function $\\mathcal P:(y_1,y_2)\\to y_1^2+y_2^2$, find an inverse function $f$ such that $\\mathcal P(f(x)) = x$ for all $x$ in $[0,1]$.\n",
"\n", "\n",

6
overview-burgers-forw.ipynb
View File

@ -8,14 +8,14 @@
"\n", "\n",
"This chapter will give an introduction for how to run _forward_, i.e., regular simulations starting with a given initial state and approximating a later state numerically, with Φ<sub>Flow</sub>. The Φ<sub>Flow</sub> framework provides a set of differentiable building blocks that directly interface with deep learning frameworks, and before going for deeper and more complicated integrations, this notebook (and the next one), will show how regular simulations can be done with Φ<sub>Flow</sub>. Later on, we can use very similar code to couple them with neural networks.\n", "This chapter will give an introduction for how to run _forward_, i.e., regular simulations starting with a given initial state and approximating a later state numerically, with Φ<sub>Flow</sub>. The Φ<sub>Flow</sub> framework provides a set of differentiable building blocks that directly interface with deep learning frameworks, and before going for deeper and more complicated integrations, this notebook (and the next one), will show how regular simulations can be done with Φ<sub>Flow</sub>. Later on, we can use very similar code to couple them with neural networks.\n",
"\n", "\n",
"The main repository for Φ<sub>Flow</sub> (in the following \"phiflow\") is https://github.com/tum-pbs/PhiFlow, and additional API documenation and examples can be found at https://tum-pbs.github.io/PhiFlow/.\n", "The main repository for Φ<sub>Flow</sub> (in the following \"phiflow\") is https://github.com/tum-pbs/PhiFlow, and additional API documentation and examples can be found at https://tum-pbs.github.io/PhiFlow/.\n",
"\n", "\n",
"For this jupyter notebook (and all following ones), you can find a _\"[run in colab]\"_ link at the end of the first paragraph. This link will load the latest version from the PBDL github repo in a colab notebook that you can execute on the spot: \n", "For this jupyter notebook (and all following ones), you can find a _\"[run in colab]\"_ link at the end of the first paragraph. This link will load the latest version from the PBDL github repo in a colab notebook that you can execute on the spot: \n",
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/overview-burgers-forw.ipynb)\n", "[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/overview-burgers-forw.ipynb)\n",
"\n", "\n",
"## Model\n", "## Model\n",
"\n", "\n",
"As phyical model we'll use Burgers equation.\n", "As physical model we'll use Burgers equation.\n",
"This equation is a very simple, yet non-linear and non-trivial, model equation that can lead to interesting shock formations. Hence, it's a very good starting point for experiments, and it's 1D version (from equation {eq}`model-burgers1d`) is given by:\n", "This equation is a very simple, yet non-linear and non-trivial, model equation that can lead to interesting shock formations. Hence, it's a very good starting point for experiments, and it's 1D version (from equation {eq}`model-burgers1d`) is given by:\n",
"\n", "\n",
"$$\n", "$$\n",
@ -270,7 +270,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"This concludes a first simulation in phiflow. It's not overly complex, but because of that it's a good starting point for evaluating and comparing different physics-based deep learning approaches in the next chatper. But before that, we'll target a more complex simulation type in the next section." "This concludes a first simulation in phiflow. It's not overly complex, but because of that it's a good starting point for evaluating and comparing different physics-based deep learning approaches in the next chapter. But before that, we'll target a more complex simulation type in the next section."
] ]
}, },
{ {

8
overview-ns-forw.ipynb
View File

@ -8,7 +8,7 @@
"source": [ "source": [
"# Navier-Stokes Forward Simulation\n", "# Navier-Stokes Forward Simulation\n",
"\n", "\n",
"Now let's target a somewhat more complex example: a fluid simulations based on the Navier-Stokes equations. This is still very simple with Φ<sub>Flow</sub> (phiflow), as differentiable operators for all steps exist. The Navier-Stokes equations (in their incompressible form) introduce an additional pressure field $p$, and a constraint for conservation of mass, as introduced in equation {eq}`model-boussinesq2d`. We're also moving a marker field, denoted by $d$ here, with the flow. It indicates regions of higher temperature, and excerts a force via with a bouyancy factor $\\xi$:\n", "Now let's target a somewhat more complex example: a fluid simulation based on the Navier-Stokes equations. This is still very simple with Φ<sub>Flow</sub> (phiflow), as differentiable operators for all steps exist. The Navier-Stokes equations (in their incompressible form) introduce an additional pressure field $p$, and a constraint for conservation of mass, as introduced in equation {eq}`model-boussinesq2d`. We're also moving a marker field, denoted by $d$ here, with the flow. It indicates regions of higher temperature, and exerts a force via a buouyancy factor $\\xi$:\n",
"\n", "\n",
"$$\\begin{aligned}\n", "$$\\begin{aligned}\n",
" \\frac{\\partial \\mathbf{u}}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla \\mathbf{u} &= - \\frac{\\Delta t}{\\rho} \\nabla p + \\nu \\nabla\\cdot \\nabla \\mathbf{u} + (0,1)^T \\xi d\n", " \\frac{\\partial \\mathbf{u}}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla \\mathbf{u} &= - \\frac{\\Delta t}{\\rho} \\nabla p + \\nu \\nabla\\cdot \\nabla \\mathbf{u} + (0,1)^T \\xi d\n",
@ -18,7 +18,7 @@
"\\end{aligned}$$\n", "\\end{aligned}$$\n",
"\n", "\n",
"\n", "\n",
"Here $\\mathbf{g}$ collects the forcing terms. Below we'll use a simple buoyancy model. We'll solve this PDE on a closed domain with Dirchlet boundary conditions $\\mathbf{u}=0$ for the velocity, and Neumann boundaries $\\frac{\\partial p}{\\partial x}=0$ for pressure, on a domain $\\Omega$ with a physical size of $100 \\times 80$ units. \n", "Here $\\mathbf{g}$ collects the forcing terms. Below we'll use a simple buoyancy model. We'll solve this PDE on a closed domain with Dirichlet boundary conditions $\\mathbf{u}=0$ for the velocity, and Neumann boundaries $\\frac{\\partial p}{\\partial x}=0$ for pressure, on a domain $\\Omega$ with a physical size of $100 \\times 80$ units. \n",
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/overview-ns-forw.ipynb)\n", "[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/overview-ns-forw.ipynb)\n",
"\n" "\n"
] ]
@ -47,7 +47,7 @@
"#!pip install --upgrade --quiet git+https://github.com/tum-pbs/PhiFlow@develop\n", "#!pip install --upgrade --quiet git+https://github.com/tum-pbs/PhiFlow@develop\n",
"#!pip install --upgrade --quiet phiflow \n", "#!pip install --upgrade --quiet phiflow \n",
"\n", "\n",
"from phi.flow import * # The Dash GUI is not supported on Google Colab, ignore the warning\n", "from phi.flow import * # The Dash GUI is not supported on Google colab, ignore the warning\n",
"import pylab" "import pylab"
] ]
}, },
@ -495,7 +495,7 @@
"source": [ "source": [
"## Next steps\n", "## Next steps\n",
"\n", "\n",
"You could create a variety of nice fluid simulations based on this setup. E.g., try chaning the spatial resolution, the buoyancy factors, and the overall length of the simulation run." "You could create a variety of nice fluid simulations based on this setup. E.g., try changing the spatial resolution, the buoyancy factors, and the overall length of the simulation run."
] ]
} }
], ],

2
overview.md
View File

@ -81,7 +81,7 @@ what can be done with numerical methods: e.g., in scenarios
where a solver targets cases from a certain well-defined problem where a solver targets cases from a certain well-defined problem
domain repeatedly, it can make a lot of sense to once invest domain repeatedly, it can make a lot of sense to once invest
significant resources to train significant resources to train
an neural network that supports the repeated solves. Based on the a neural network that supports the repeated solves. Based on the
domain-specific specialization of this network, such a hybrid domain-specific specialization of this network, such a hybrid
could vastly outperform traditional, generic solvers. And despite could vastly outperform traditional, generic solvers. And despite
the many open questions, first publications have demonstrated the many open questions, first publications have demonstrated

4
supervised-airfoils.ipynb
View File

@ -646,7 +646,7 @@
"\n", "\n",
"Now let's look at actual test samples: In this case we'll use new airfoil shapes as out-of-distribution (OOD) data. These are shapes that the network never saw in any training samples, and hence it tells us a bit about how well the model generalizes to unseen inputs (the validation data wouldn't suffice to draw conclusions about generalization).\n", "Now let's look at actual test samples: In this case we'll use new airfoil shapes as out-of-distribution (OOD) data. These are shapes that the network never saw in any training samples, and hence it tells us a bit about how well the model generalizes to unseen inputs (the validation data wouldn't suffice to draw conclusions about generalization).\n",
"\n", "\n",
"We'll use the samme visualization as before, and as indicated by the Bernoulli equation, especially the _pressure_ in the first column is a challenging quantity for the network. Due to it's cubic scaling w.r.t. the input freestream velocity and localized peaks, it is the toughest quantity to infer for the network.\n", "We'll use the same visualization as before, and as indicated by the Bernoulli equation, especially the _pressure_ in the first column is a challenging quantity for the network. Due to it's cubic scaling w.r.t. the input freestream velocity and localized peaks, it is the toughest quantity to infer for the network.\n",
"\n", "\n",
"The cell below first downloads a smaller archive with these test data samples, and then runs them through the network. The evaluation loop also computes the accumulated L1 error such that we can quantify how well the network does on the test samples." "The cell below first downloads a smaller archive with these test data samples, and then runs them through the network. The evaluation loop also computes the accumulated L1 error such that we can quantify how well the network does on the test samples."
] ]
@ -781,7 +781,7 @@
"id": "vhH-rUZ-JMTX" "id": "vhH-rUZ-JMTX"
}, },
"source": [ "source": [
"## Nex steps\n", "## Next steps\n",
"\n", "\n",
"There are many obvious things to try here (see the suggestions below), e.g. longer training, larger data sets, larger networks etc. \n", "There are many obvious things to try here (see the suggestions below), e.g. longer training, larger data sets, larger networks etc. \n",
"\n", "\n",

2
supervised-discuss.md
View File

@ -30,7 +30,7 @@ To summarize the scattered comments of the previous sections, here's a set of "g
- Always start with a 1-sample overfitting test. - Always start with a 1-sample overfitting test.
- Check how many trainable parameters your network has. - Check how many trainable parameters your network has.
- Slowly increase the amount of trianing data (and potentially network parameters and depth). - Slowly increase the amount of training data (and potentially network parameters and depth).
- Adjust hyperparameters (especially the learning rate). - Adjust hyperparameters (especially the learning rate).
- Then introduce other components such as differentiable solvers or adversarial training. - Then introduce other components such as differentiable solvers or adversarial training.