updates and teaks flow matching and continuity across the whole chapter
This commit is contained in:
parent
6391dbab10
commit
4589cf2860
@ -8,7 +8,7 @@
|
||||
"source": [
|
||||
"# Denoising and Flow Matching Side-by-side\n",
|
||||
"\n",
|
||||
"To show the capabilities of **denoising diffusion** and **flow matching**, we'll be use a learning task where we can reliably generate arbitrary amounts of ground truth data. This ensures we can quantify how well the target distribution was learned. Specifically, we'll focus on Reynolds-averaged Navier-Stokes simulations around airfoils, which have the interesting characteristic that typical solvers (such as OpenFoam) transition from steady solutions to oscillating ones for larger Reynolds numbers. This transition is exactly what we'll give as a task to diffusion models below. (Details can be found in our [diffuion-based flow prediction repository](https://github.com/tum-pbs/Diffusion-based-Flow-Prediction/).) Also, to make the notebook self-contained, we'll revisit the most important concepts from the previous section.\n",
|
||||
"To show the capabilities of **denoising diffusion** and **flow matching**, we'll be use a learning task where we can reliably generate arbitrary amounts of ground truth data. This ensures we can quantify how well the target distribution was learned. Specifically, we'll focus on Reynolds-averaged Navier-Stokes simulations around airfoils, which have the interesting characteristic that typical solvers (such as OpenFoam) transition from steady solutions to oscillating ones for larger Reynolds numbers. This transition is exactly what we'll give as a task to diffusion models below. (Details can be found in our [diffusion-based flow prediction repository](https://github.com/tum-pbs/Diffusion-based-Flow-Prediction/).) Also, to make the notebook self-contained, we'll revisit the most important concepts from the previous section.\n",
|
||||
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/probmodels-ddpm-fm.ipynb)\n",
|
||||
"\n",
|
||||
"```{note} \n",
|
||||
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@ -750,14 +750,21 @@
|
||||
"\n",
|
||||
"The way in which we can achieve this are _Neural ODEs_{cite}`chen2019node`.\n",
|
||||
"They're especially interesting in the context of physics simulations. Mathematically speaking, they replace the mapping $g_i$ with a learned velocity predictor $g_\\theta$. This means\n",
|
||||
"$$\\frac{\\partial z(t)}{\\partial t} = g_\\theta(z(t), t).$$\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{\\partial z(t)}{\\partial t} = g_\\theta(z(t), t).\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"Then the sequence of $g_i$ steps that made up $g$ can be replaced by integrating the velocity, which gives a simple ODE integration.\n",
|
||||
"The continuous time axis is introduced, with $t=0$ starting at a normal Gaussian distribution, to $t=1$ for the target distribution.\n",
|
||||
"The ODE is solved along this timeline, e.g. to transform the base distribution $p_Z$ into the target $p_Y$ by querying $g$ for a velocity at each time point along the way.\n",
|
||||
"Even better, for an ODE solve there's an analytic formulation for the gradient of the backpropagation path. This is a neat example of a differentiable physics solver (the ODE solve), providing an efficient way to compute a gradient, and aligns with the topics discussed in {doc}`physgrad`.\n",
|
||||
"\n",
|
||||
"The change in probabilities over time can also be computed conveniently via the trace of the learned function:\n",
|
||||
"$$\\frac{\\partial \\log p(z(t))}{\\partial t} = - \\mathrm{Tr} \\left( \\frac{\\partial g_\\theta}{\\partial z(t)} \\right).$$\n",
|
||||
"\n",
|
||||
"$$\n",
|
||||
"\\frac{\\partial \\log p(z(t))}{\\partial t} = - \\mathrm{Tr} \\left( \\frac{\\partial g_\\theta}{\\partial z(t)} \\right).\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"Compared to the Normalizing Flows above, an important difference in the NeuralODE picture is that now we have a single function $g_\\theta(\\cdot,t)$ that is repeatedly evaluated at different points in the time interval $t \\in [0,1]$. This might seem like a trivial change at first, but it's a crucial step towards more powerful probabilistic models such as diffusion models.\n",
|
||||
"It turns out to be important that we can re-use a learned function, instead of having to manually construct many layers with large numbers of trainable parameters.\n",
|
||||
|
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue
Block a user