daviddoji/pbdl-book
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

additional corrections teaser and overview

Browse Source
This commit is contained in:
NT 2021-07-10 10:11:53 +02:00
parent 980e65f47f
commit c8feb79fe7
5 changed files with 50 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
intro-teaser.ipynb
View File

File diff suppressed because one or more lines are too long

12
overview-burgers-forw.ipynb
View File

@ -6,9 +6,9 @@
"source": [
"# Simple Forward Simulation of Burgers Equation with phiflow\n",
"\n",
"This chapter will give an introduction for how to run _forward_, i.e., regular simulations starting with a given initial state and approximating a later state numerically, with Φ<sub>Flow</sub>. The Φ<sub>Flow</sub> framework provides a set of differentiable building blocks that directly interface with deep learning frameworks, and before going for deeper and more complicated integrations, this notebook (and the next one), will show how regular simulations can be done with Φ<sub>Flow</sub>. Later on, we can use very similar code to couple them with neural networks.\n",
"This chapter will give an introduction for how to run _forward_, i.e., regular simulations starting with a given initial state and approximating a later state numerically, with Φ<sub>Flow</sub>. The Φ<sub>Flow</sub> framework provides a set of differentiable building blocks that directly interface with deep learning frameworks. Before going for deeper and more complicated integrations, this notebook (and the next one), will show how regular simulations can be done with Φ<sub>Flow</sub>. Later on, we can use very similar code to couple them with neural networks.\n",
"\n",
"The main repository for Φ<sub>Flow</sub> (in the following \"phiflow\") is https://github.com/tum-pbs/PhiFlow, and additional API documentation and examples can be found at https://tum-pbs.github.io/PhiFlow/.\n",
"The main repository for Φ<sub>Flow</sub> (in the following \"phiflow\") is [https://github.com/tum-pbs/PhiFlow](https://github.com/tum-pbs/PhiFlow), and additional API documentation and examples can be found at [https://tum-pbs.github.io/PhiFlow/](https://tum-pbs.github.io/PhiFlow/).\n",
"\n",
"For this jupyter notebook (and all following ones), you can find a _\"[run in colab]\"_ link at the end of the first paragraph. This link will load the latest version from the PBDL github repo in a colab notebook that you can execute on the spot: \n",
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/overview-burgers-forw.ipynb)\n",
@ -50,7 +50,7 @@
}
],
"source": [
"#!pip install --upgrade --quiet phiflow\n",
"!pip install --upgrade --quiet phiflow\n",
"\n",
"from phi.flow import *\n",
"\n",
@ -130,7 +130,7 @@
"source": [
"## Running the simulation\n",
"\n",
"Now we're ready to run the simulation itself. To compute the diffusion and advection components of our model equation we can simply call the existing `diffusion` and `semi_lagrangian` operators in phiflow:"
"Now we're ready to run the simulation itself. To ccompute the diffusion and advection components of our model equation we can simply call the existing `diffusion` and `semi_lagrangian` operators in phiflow: `diffuse.explicit(u,...)` computes an explicit diffusion step via central differences for the term $\\nu \\nabla\\cdot \\nabla u$ of our model. Next, `advect.semi_lagrangian(f,u)` is used for a stable first-order approximation of the transport of an arbitrary field `f` by a velocity `u`. In our model we have $\\partial u / \\partial{t} + u \\nabla f$, hence we use the `semi_lagrangian` function to transport the velocity with itself in the implementation:"
]
},
{
@ -164,7 +164,7 @@
"source": [
"Here we're actually collecting all time steps in the list `velocities`. This is not necessary in general (and could consume lots of memory for long-running sims), but useful here to plot the evolution of the velocity states later on.\n",
"\n",
"The print statements print a few of the velocity entries, and already shows that something is happening in our simulation, but it's difficult to get an intuition for the behavior of the PDE just from these numbers. Hence, let's visualize the states over time to show what is happening.\n",
"The print statements print a few of the velocity entries, and already show that something is happening in our simulation, but it's difficult to get an intuition for the behavior of the PDE just from these numbers. Hence, let's visualize the states over time to show what is happening.\n",
"\n",
"## Visualization\n",
"\n",
@ -221,7 +221,7 @@
"\n",
"As these lines can overlap quite a bit we'll also use a different visualization in the following chapters that shows the evolution over the course of all time steps in a 2D image. Our 1D domain will be shown along the Y-axis, and each point along X will represent one time step.\n",
"\n",
"The code below converts our collection of velocity states into a 2D array, repeating individual time steps 8 times to make the image a bit wider. This purely optional, of course, but makes it easier to see what's happening in our Burgers simulation."
"The code below converts our collection of velocity states into a 2D array, repeating individual time steps 8 times to make the image a bit wider. This is purely optional, of course, but makes it easier to see what's happening in our Burgers simulation."
]
},
{

26
overview-equations.md
View File

@ -27,7 +27,7 @@ $$
$$ (learn-l2)
We typically optimize, i.e. _train_,
with a stochastic gradient descent (SGD) optimizer of your choice, e.g., Adam {cite}`kingma2014adam`.
with a stochastic gradient descent (SGD) optimizer of your choice, e.g. Adam {cite}`kingma2014adam`.
We'll rely on auto-diff to compute the gradient w.r.t. weights, $\partial f / \partial \theta$,
We will also assume that $e$ denotes a _scalar_ error function (also
called cost, or objective function).
@ -78,7 +78,7 @@ positions are denoted by $\mathbf{x} \in \Omega$.
To obtain unique solutions for $\mathcal P^*$ we need to specify suitable
initial conditions, typically for all quantities of interest at $t=0$,
and boundary conditions for the boundary or $\Omega$, denoted by $\Gamma$ in
and boundary conditions for the boundary of $\Omega$, denoted by $\Gamma$ in
the following.
$\mathcal P^*$ denotes
@ -87,7 +87,7 @@ its continuity, we will typically assume that first and second derivatives exist
We can then use numerical methods to obtain approximations
of a smooth function such as $\mathcal P^*$ via discretization.
This invariably introduce discretization errors, which we'd like to keep as small as possible.
These invariably introduce discretization errors, which we'd like to keep as small as possible.
These errors can be measured in terms of the deviation from the exact analytical solution,
and for discrete simulations of PDEs, they are typically expressed as a function of the truncation error
$O( \Delta x^k )$, where $\Delta x$ denotes the spatial step size of the discretization.
@ -96,7 +96,7 @@ Likewise, we typically have a temporal discretization via a time step $\Delta t$
```{admonition} Notation and abbreviations
:class: seealso
If unsure, please check the summary of our mathematical notation
and the abbreviations used inn: {doc}`notation`.
and the abbreviations used in: {doc}`notation`.
```
% \newcommand{\pde}{\mathcal{P}} % PDE ops
@ -186,15 +186,19 @@ In 2D, the Navier-Stokes equations without any external forces can be written as
$$\begin{aligned}
\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &=
- \frac{1}{\rho}\nabla{p} + \nu \nabla\cdot \nabla u_x
- \frac{\Delta t}{\rho}\nabla{p} + \nu \nabla\cdot \nabla u_x
\\
\frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &=
- \frac{1}{\rho}\nabla{p} + \nu \nabla\cdot \nabla u_y
- \frac{\Delta t}{\rho}\nabla{p} + \nu \nabla\cdot \nabla u_y
\\
\text{subject to} \quad \nabla \cdot \mathbf{u} &= 0
\end{aligned}$$ (model-ns2d)
where, like before, $\nu$ denotes a diffusion constant for viscosity.
In practice, the $\Delta t$ factor for the pressure term can be often simplified to
$1/\rho$ as it simply yields a scaling of the pressure gradient used to make
the velocity divergence free. We'll typically use this simplification later on
in implementations, effectively computing an instantaneous pressure.
An interesting variant is obtained by including the
[Boussinesq approximation](https://en.wikipedia.org/wiki/Boussinesq_approximation_(buoyancy))
@ -203,9 +207,9 @@ With a marker field $v$, e.g., indicating regions of high temperature,
this yields the following set of equations:
$$\begin{aligned}
\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &= - \frac{1}{\rho} \nabla p
\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &= - \frac{\Delta t}{\rho} \nabla p
\\
\frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &= - \frac{1}{\rho} \nabla p + \xi v
\frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &= - \frac{\Delta t}{\rho} \nabla p + \xi v
\\
\text{subject to} \quad \nabla \cdot \mathbf{u} &= 0,
\\
@ -218,11 +222,11 @@ And finally, the Navier-Stokes model in 3D give the following set of equations:
$$
\begin{aligned}
\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &= - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_x
\frac{\partial u_x}{\partial{t}} + \mathbf{u} \cdot \nabla u_x &= - \frac{\Delta t}{\rho} \nabla p + \nu \nabla\cdot \nabla u_x
\\
\frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &= - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_y
\frac{\partial u_y}{\partial{t}} + \mathbf{u} \cdot \nabla u_y &= - \frac{\Delta t}{\rho} \nabla p + \nu \nabla\cdot \nabla u_y
\\
\frac{\partial u_z}{\partial{t}} + \mathbf{u} \cdot \nabla u_z &= - \frac{1}{\rho} \nabla p + \nu \nabla\cdot \nabla u_z
\frac{\partial u_z}{\partial{t}} + \mathbf{u} \cdot \nabla u_z &= - \frac{\Delta t}{\rho} \nabla p + \nu \nabla\cdot \nabla u_z
\\
\text{subject to} \quad \nabla \cdot \mathbf{u} &= 0.
\end{aligned}

10
overview-ns-forw.ipynb
View File

@ -11,14 +11,15 @@
"Now let's target a somewhat more complex example: a fluid simulation based on the Navier-Stokes equations. This is still very simple with Φ<sub>Flow</sub> (phiflow), as differentiable operators for all steps exist. The Navier-Stokes equations (in their incompressible form) introduce an additional pressure field $p$, and a constraint for conservation of mass, as introduced in equation {eq}`model-boussinesq2d`. We're also moving a marker field, denoted by $d$ here, with the flow. It indicates regions of higher temperature, and exerts a force via a buouyancy factor $\\xi$:\n",
"\n",
"$$\\begin{aligned}\n",
" \\frac{\\partial \\mathbf{u}}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla \\mathbf{u} &= - \\frac{\\Delta t}{\\rho} \\nabla p + \\nu \\nabla\\cdot \\nabla \\mathbf{u} + (0,1)^T \\xi d\n",
" \\frac{\\partial \\mathbf{u}}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla \\mathbf{u} &= - \\frac{1}{\\rho} \\nabla p + \\nu \\nabla\\cdot \\nabla \\mathbf{u} + (0,1)^T \\xi d\n",
" \\quad \\text{s.t.} \\quad \\nabla \\cdot \\mathbf{u} = 0,\n",
" \\\\\n",
" \\frac{\\partial d}{\\partial{t}} + \\mathbf{u} \\cdot \\nabla d &= 0 \n",
"\\end{aligned}$$\n",
"\n",
"\n",
"Here $\\mathbf{g}$ collects the forcing terms. Below we'll use a simple buoyancy model. We'll solve this PDE on a closed domain with Dirichlet boundary conditions $\\mathbf{u}=0$ for the velocity, and Neumann boundaries $\\frac{\\partial p}{\\partial x}=0$ for pressure, on a domain $\\Omega$ with a physical size of $100 \\times 80$ units. \n",
"Here, we're aiming for an incompressible flow (i.e., $\\rho = \\text{const}$), and use a simple buoyancy model (the Boussinesq approximation) via the term $(0,1)^T \\xi d$. This models changes in density without explicitly calculating $\\rho$, and we assume a gravity force that acts along the y direction via the vector $(0,1)^T$. \n",
"We'll solve this PDE on a closed domain with Dirichlet boundary conditions $\\mathbf{u}=0$ for the velocity, and Neumann boundaries $\\frac{\\partial p}{\\partial x}=0$ for pressure, on a domain $\\Omega$ with a physical size of $100 \\times 80$ units. \n",
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/overview-ns-forw.ipynb)\n",
"\n"
]
@ -84,7 +85,7 @@
"id": "ExA0Pi2sFVka"
},
"source": [
"The inflow will be used to inject smoke into a second centered grid `smoke` grid that represents the marker field $d$ from above. Note that we've defined a `Box` of size $100x80$ above. This is the physical scale in terms of spatial units in our simulation, i.e., a velocity of magnitude $1$ will move the smoke density by 1 unit per 1 time unit, which may be larger or smaller than a cell in the discretized grid, depending on the settings for `x,y`. You could parametrize your simulation grid to directly resemble real-world units, or keep conversion factors in mind for conversion. \n",
"The inflow will be used to inject smoke into a second centered grid `smoke` that represents the marker field $d$ from above. Note that we've defined a `Box` of size $100x80$ above. This is the physical scale in terms of spatial units in our simulation, i.e., a velocity of magnitude $1$ will move the smoke density by 1 unit per 1 time unit, which may be larger or smaller than a cell in the discretized grid, depending on the settings for `x,y`. You could parametrize your simulation grid to directly resemble real-world units, or keep appropriate conversion factors in mind. \n",
"\n",
"The inflow sphere above is already using the \"world\" coordinates: it is located at $x=30$ along the first axis, and $y=15$ (within the $100x80$ domain box).\n",
"\n",
@ -174,7 +175,8 @@
"source": [
"A lot has happened in this `step()` call: we've advected the smoke field, added an upwards force via a Boussinesq model, advected the velocity field, and finally made it divergence free via a pressure solve.\n",
"\n",
"The Boussinesq model uses a multiplication by a tuple `(0, buoyancy_factor)` to turn the smoke field into a 2 component force field, sampled at the staggered velocity components via the `>>` operator. **TODO, write out**\n",
"The Boussinesq model uses a multiplication by a tuple `(0, buoyancy_factor)` to turn the smoke field into a staggered, 2 component force field, sampled at the locations of the velocity components via the `>>` operator. \n",
"This operator makes sure the individual force components are correctly interpolated for the velocity components of the staggered velocity. `>>` could be rewritten via the sampling function `at()` as `(smoke*(0,buoyancy_factor)).at(velocity)`. However, `>>` also directly ensure the boundary conditions of the original grid are kept. Hence, it internally also does `StaggeredGrid(..., extrapolation.ZERO,...)` for the force grid. Long story short: the `>>` operator does the same thing, and typically produces shorter and more readable code.\n",
"\n",
"The pressure projection step in `make_incompressible` is typically the computationally most expensive step in the sequence above. It solves a Poisson equation for the boundary conditions of the domain, and updates the velocity field with the gradient of the computed pressure.\n",
"\n",

16
overview.md
View File

@ -5,8 +5,8 @@ The name of this book, _Physics-Based Deep Learning_,
denotes combinations of physical modeling and numerical simulations with
methods based on artificial neural networks.
The general direction of Physics-Based Deep Learning represents a very
active, quickly growing and exciting field of research, and the following chapter will
give a more thorough introduction for the topic and establish the basics
active, quickly growing and exciting field of research. The following chapter will
give a more thorough introduction to the topic and establish the basics
for following chapters.
```{figure} resources/overview-pano.jpg
@ -32,14 +32,14 @@ have led to impressive achievements in a variety of fields:
from image classification {cite}`krizhevsky2012` over
natural language processing {cite}`radford2019language`,
and more recently also for protein folding {cite}`alquraishi2019alphafold`.
The field is very vibrant, and quickly developing, with the promise of vast possibilities.
The field is very vibrant and quickly developing, with the promise of vast possibilities.
On the other hand, the successes of deep learning (DL) approaches
has given rise to concerns that this technology has
have given rise to concerns that this technology has
the potential to replace the traditional, simulation-driven approach to
science. Instead of relying on models that are carefully crafted
from first principles, can data collections of sufficient size
be processed to provide the correct answers instead?
be processed to provide the correct answers?
In short: this concern is unfounded. As we'll show in the next chapters,
it is crucial to bring together both worlds: _classical numerical techniques_
and _deep learning_.
@ -77,9 +77,9 @@ I.e., our goal is to _reconcile_ the data-centered
viewpoint and the physical simulation viewpoint.
The resulting methods have a huge potential to improve
what can be done with numerical methods: e.g., in scenarios
what can be done with numerical methods: in scenarios
where a solver targets cases from a certain well-defined problem
domain repeatedly, it can make a lot of sense to once invest
domain repeatedly, it can for instance make a lot of sense to once invest
significant resources to train
a neural network that supports the repeated solves. Based on the
domain-specific specialization of this network, such a hybrid
@ -105,7 +105,7 @@ observations).
![An overview of categories of physics-based deep learning methods](resources/physics-based-deep-learning-overview.jpg)
No matter whether we're considering forward or inverse problem,
No matter whether we're considering forward or inverse problems,
the most crucial differentiation for the following topics lies in the
nature of the integration between DL techniques
and the domain knowledge, typically in the form of model equations