updated teaser, added dividers

diffphys-discuss.md

@@ -8,6 +8,8 @@ achieved via DP approaches it is nonetheless a good time to discuss some
 additional properties, and summarize the pros and cons.
 
 
+![Divider](resources/divider4.jpg)
+
 ## Time Steps and Iterations
 
 When using DP approaches for learning application, there is a large amount of flexibility
@@ -73,6 +75,8 @@ for training that tends to overfit, but if possible, it is preferable to incorpo
 actual solver in the training loop via a DP approach.
 
 
+![Divider](resources/divider5.jpg)
+
 ## Summary
 
 To summarize the pros and cons of training NNs via DP:

diffphys-outlook.md

@@ -3,6 +3,8 @@ Summary and Discussion
 
 The previous sections have explained the differentiable physics approach for deep learning, and have given a range of examples: from a very basic gradient calculation, all the way to complex learning setups powered by complex simulations. This is a good time to pause and take a step back, to take a look at what we have: in the end, the _differentiable physics_ component of these approaches is not too complicated. It's largely based on existing numerical methods, with a focus on efficiently using those methods to not only do a forward simulation, but also to compute gradient information. What is primarily exciting in this context are the implications that arise from the combination of these numerical methods with deep learning.
 
+![Divider](resources/divider6.jpg)
+
 ## Integration
 
 Most importantly, training via differentiable physics allows us to seamlessly bring the two fields together:
@@ -16,7 +18,7 @@ One key aspect that is important for these hybrids to work well is to let the NN
 
 The hybrid approach also bears particular promise for simulators: it improves generalizing capabilities of the trained models by letting the PDE-solver handle large-scale _changes to the data distribution_ such that the learned model can focus on localized structures not captured by the discretization. While physical models generalize very well, learned models often specialize in data distributions seen at training time. This was, e.g., shown for the models reducing numerical errors of the previous chapter: the trained models can deal with solution manifolds with significant amounts of varying physical behavior, while simpler training variants quickly deteriorate over the course of recurrent time steps.
 
----
+![Divider](resources/divider7.jpg)
 
 Training NNs via differentiable physics solvers, i.e., what we've described as "DP" in the previous
 sections, is a very generic approach that is applicable to a wide range of combinations of PDE-based models

intro-teaser.ipynb

@@ -390,10 +390,17 @@
    "source": [
     "## Discussion\n",
     "\n",
-    "It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and often it's not clear where (and how many) exist in the solution manifold we're interested in. Then supervised learning is very dangerous - we might simply and unknowingly _blur_ out these different modes.\n",
+    "It's a very simple example, but it very clearly shows a failure case for supervised learning. While it might seem very artificial on first sight, many practical PDEs exhibit a variety of these modes, and often it's not clear where (and how many) exist in the solution manifold we're interested in. Using supervised learning is very dangerous in such cases - we might simply and unknowingly _blur_ out these different modes.\n",
     "\n",
     "A good examples are bifurcations in fluid flows - the smoke rising above a candle will start out straight, and then, due to tiny perturbations in its motion, start oscillating in a random direction. The images below illustrate this case via _numerical perturbations_: the perfectly symmetric setup will start turning left or right, depending on how the approximation errors build up. Similarly, we'll have different modes in all our numerical solutions, and typically it's important to recover them, rather than averaging out. Hence, we'll show how to leverage training via _differentiable physics_ in the following chapters for more practical and complex cases.\n",
-    "\n"
+    "\n",
+    "```{figure} resources/intro-fluid-bifurcation.jpg\n",
+    "---\n",
+    "height: 240px\n",
+    "name: intro-fluid-bifurcation \n",
+    "---\n",
+    "A bifurcation in a buoyancy-driven fluid flow: the \"smoke\" shown in green color starts rising in a percectly straight manner, but tiny numerical inaccuracies grow over time to lead to an instability with vortices alternating to one side (top-right), or in the opposite direction (bottom-right). \n",
+    "```\n"
    ]
   },
   {

intro.md

@@ -6,8 +6,9 @@ Welcome to the _Physics-based Deep Learning Book_ 👋
 **TL;DR**: 
 This document targets a variety of combinations of physical simulations with deep learning.
 As much as possible, the algorithms will come with hands-on code examples to quickly get started.
-Beyond standard _supervised_ learning from data, we'll look at _physical loss_ constraints, and 
+Beyond standard _supervised_ learning from data, we'll look at _physical loss_ constraints, 
-more tightly coupled learning algorithms with _differentiable simulations_.
+more tightly coupled learning algorithms with _differentiable simulations_, as well as extensions such
+as reinforcement learning and uncertainty modeling.
 
 
 ```{figure} resources/teaser.jpg
@@ -17,8 +18,6 @@ name: pbdl-teaser
 ---
 Some visual examples of numerically simulated time sequences. In this book, we aim for algorithms that use neural networks alongside numerical solvers.
 ```
-% Teaser, simple version:
-% ![Teaser, simple version](resources/teaser.jpg)
 
 ## Coming up
 
@@ -32,8 +31,8 @@ As a _sneak preview_, in the next chapters will show:
 
 This _book_, where "book" stands for a collection of texts, equations, images and code examples,
 is maintained by the
-[TUM Physics-based Simulation Group](https://ge.in.tum.de). Feel free to contact us via
+[TUM Physics-based Simulation Group](https://ge.in.tum.de). Feel free to contact us 
-[old fashioned email](mailto:i15ge@cs.tum.edu) if you have any comments. 
+if you have any comments, e.g., via [old fashioned email](mailto:i15ge@cs.tum.edu).
 If you find mistakes, please also let us know! We're aware that this document is far from perfect,
 and we're eager to improve it. Thanks in advance!
 
@@ -51,7 +50,7 @@ Oh, and it's great because it's [literate programming](https://en.wikipedia.org/
 ```
 
 
----
+![Divider](resources/divider3.jpg)
 
 
 ## Thanks!

outlook.md

@@ -5,6 +5,8 @@ Despite the lengthy discussions and numerous examples,
 we've really just barely scratched the surface regarding the possibilities that arise in the context 
 of physics-based deep learning.
 
+![Divider](resources/divider2.jpg)
+
 The examples with Burgers equation and Navier-Stokes solvers are non-trivial, and good examples for advection-diffusion-type PDEs. However, there's a wide variety of other potential combinations. To name just a few promising examples from other fields:
 
 * PDEs for chemical reactions often show complex behavior due to the interactions of multiple species. Here, and especially interesting direction is to train models that quickly learn to predict the evolution of an experiment or machine, and adjust control knobs to stabilize it, i.e., an online _control_ setting.
@@ -14,3 +16,6 @@ The examples with Burgers equation and Navier-Stokes solvers are non-trivial, an
 * Finally, weather and climate are crucial topics for humanity, and highly complex systems of fluid flows interacting with a multitude of phenomena on the surface of our planet. Accurately modeling all these interacting systems and predicting their long-term behavior shows a lot of promise to benefit from DL approaches that can interface with numerical simulations.
 
 So overall, there's lots of exciting research work left to do - the next years and decades definitely won't be boring 👍
+
+![Divider](resources/divider3.jpg)
+

overview-ns-forw.ipynb

@@ -452,7 +452,7 @@
    "source": [
     "It looks simple here, but this simulation setup is a powerful tool. The simulation could easily be extended to more complex cases or 3D, and they're fully compatible with back-propagation pipelines of deep learning frameworks. \n",
     "\n",
-    "In the next chapters we'll show how to use these simulations for training NNs, and how to steer and modify them via trained NNs. This will illustrate how much we can improve the training process by having a solver in the loop, and especially by having differentiable solvers."
+    "In the next chapters we'll show how to use these simulations for training NNs, and how to steer and modify them via trained NNs. This will illustrate how much we can improve the training process by having a solver in the loop, and especially by having differentiable solvers. Before moving to these more complex training processes, we will cover a simpler supervised approach in the next chapter. This is very fundamental: even when aiming for advanced physics-based learning setups, a working supervised training is always the first step."
    ]
   },
   {

physgrad-discuss.md

@@ -14,6 +14,8 @@ by, e.g., training longer with a simpler approach. So, if we plan to evaluate th
 models often (e.g., ship them in an application), this increased one-time cost
 can pay off in the long run.
 
+![Divider](resources/divider1.jpg)
+
 ## Summary
 
 ✅ Pro: 

physicalloss-discuss.md

@@ -42,6 +42,8 @@ there is also little hope that it will yield different solutions, or perform wel
 of the training distribution. If we're interested in a different solution, we most likely 
 have to start training the NN from scratch.
 
+![Divider](resources/divider5.jpg)
+
 ## Summary
 
 Thus, the physical soft constraints allow us to encode solutions to 

supervised-discuss.md

@@ -9,6 +9,8 @@ by these solutions, and the trained network can give us predictions
 very quickly. There are a few important points to keep in mind when 
 using supervised training.
 
+![Divider](resources/divider1.jpg)
+
 ## Some things to keep in mind...
 
 ### Natural starting point
@@ -92,7 +94,7 @@ E.g., you'll notice that CNNs typically don't need dropout, as they're nicely
 regularized by construction. For MLPs, you typically need quite a bit to
 avoid overfitting.
 
----
+![Divider](resources/divider2.jpg)
 
 ## Supervised Training in a nutshell
 

supervised.md

@@ -2,11 +2,11 @@ Supervised Training
 =======================
 
 _Supervised_ here essentially means: "doing things the old fashioned way". Old fashioned in the context of 
-deep learning (DL), of course, so it's still fairly new. Also, "old fashioned" of course also doesn't 
+deep learning (DL), of course, so it's still fairly new. Also, "old fashioned" doesn't 
-always mean bad - it's just that we'll be able to do better than simple supervised training later on. 
+always mean bad - it's just that later on we'll be able to do better than with a simple supervised training.
 
 In a way, the viewpoint of "supervised training" is a starting point for all projects one would encounter in the context of DL, and
-hence is worth studying. And although it typically yields inferior results to approaches that more tightly 
+hence is worth studying. While it typically yields inferior results to approaches that more tightly 
 couple with physics, it nonetheless can be the only choice in certain application scenarios where no good
 model equations exist.