intro update

intro.md

@@ -9,6 +9,7 @@ As much as possible, the algorithms will come with hands-on code examples to qui
 Beyond standard _supervised_ learning from data, we'll look at loss constraints, and 
 more tightly coupled learning algorithms with differentiable simulations.
 
+
 ```{figure} resources/teaser.png
 ---
 height: 220px
@@ -51,9 +52,9 @@ immediately see what happens -- give it a try...
 Oh, and it's great because it's [literate programming](https://en.wikipedia.org/wiki/Literate_programming).
 ```
 
-## Specifically
+## More Specifically
 
-To be a bit more specific, _physics_ is a huge field, we can't cover everything... 
+To be a bit more specific, _physics_ is a huge field, and we can't cover everything... 
 
 ```{note}
 For now our focus is:
@@ -69,16 +70,13 @@ For now our focus is:
 
 The contents of the following files would not have been possible without the help of many people. Here's an alphabetical list. Big kudos to everyone 🙏
 
-- Mr. X
+- [Li-wei Chen](https://ge.in.tum.de/about/dr-liwei-chen/)
-- Ms. y
+- [Philipp Holl](https://ge.in.tum.de/about/)
-- ...
+- [Patrick Schnell](https://ge.in.tum.de/about/patrick-schnell/)
+- [Nils Thuerey](https://ge.in.tum.de/about/n-thuerey/)
+- [Kiwon Um](https://ge.in.tum.de/about/kiwon/)
 
-
+<!-- % some markdown tests follow ...
-
-
-
-
-% some markdown tests follow ...
 
 ---
 
@@ -91,26 +89,30 @@ See also... Test link: {doc}`supervised`
 
 ✅  Do this , ❌  Don't do this
 
-% ----------------
+% ---------------- -->
 
 ---
 
 
-## Planned content
+## TODOs , Planned content
 
 Loose collection of notes and TODOs:
 
 General physics & dl , intro & textual overview
 
-- 	Intro phys loss example, notebook patrick
+- 	Intro phys loss example, parabola example
 
-Supervised? Airfoils? Liwei, simple example? app: optimization, shape opt w surrogates
+Supervised simple starting point
 
 -    AIAA supervised learning , idp_weissenov/201019-upd-arxiv-v2/  {cite}`thuerey2020deepFlowPred`
     skepticism? , started colab -> https://colab.research.google.com/drive/11KUe5Ybuprd7_qmNTe1nvQVUz3W6gRUo
     torch version 1.7 [upd from Liwei?]
 
-vs. PINNs [alt.: neural ODEs , PDE net?] , all using GD (optional, PINNs could use BFGS)
+- surrogates, shape opt?
+
+Physical losses 
+
+- vs. PINNs [alt.: neural ODEs , PDE net?] , all using GD (optional, PINNs could use BFGS)
     [PINNs], phiflow example -> convert to colab
 
 -    PINNs -> are unsupervised a la tompson; all DL NNs are "supervised" during learning, unsup just means not precomputed and goes through function
@@ -120,7 +122,6 @@ vs. PINNs [alt.: neural ODEs , PDE net?] , all using GD (optional, PINNs could u
 -    discuss CG solver, tompson as basic ''unsupervisedd'' example?
 
 Diff phys, start with overview of idea: gradients via autodiff, then run GD
-        (TODO include squared func Patrick?)
 
 -    illustrate and discuss gradients -> mult. for chain rule; (later: more general PG chain w func composition)
 
@@ -164,4 +165,3 @@ _Misc jupyter book TODOs_
 - Fix latex PDF output
 - How to include links to papers in the bibtex references?
 
-

overview.md

@@ -10,34 +10,46 @@ active, quickly growing and exciting field of research -- we want to provide
 a starting point for new researchers as well as a hands-on introduction into
 state-of-the-art resarch topics. 
 
+
+```{figure} resources/overview-pano.jpg
+---
+height: 240px
+name: overview-pano
+---
+Understanding our environment, and predicting how it will evolve is one of the key challenges of humankind.
+```
+
 ## Motivation
 
-From weather forecasts (? ) over X, Y, 
+From weather and climate forecasts {cite}`stocker2014climate`,
-... more ...
+over quantum physics {cite}`o2016scalable`,
-to quantum physics (? ),
+to the control of plasma fusion {cite}`maingi2019fesreport`,
 using numerical analysis to obtain solutions for physical models has
 become an integral part of science.
 
 At the same time, machine learning technologies and deep neural networks in particular,
-have led to impressive achievements in a variety of field.
+have led to impressive achievements in a variety of fields:
-Among others, GPT-3
+from image classification {cite}`krizhevsky2012` over
-has recently demonstrated that learning methods can
+natural language processing {cite}`radford2019language`, 
-achieve astounding accuracy for processing natural language.
+and more recently also for protein folding {cite}`alquraishi2019alphafold`.
-Also: AlphaGO, closer to physics: protein folding...
+The field is very vibrant, and quickly developing, with the promise of vast possibilities.
-This is a vibrant, quickly developing field with vast possibilities.
 
-The successes of DL approaches have given rise to concerns that this technology has 
+At the same time, the successes of deep learning (DL) approaches 
+has given rise to concerns that this technology has 
 the potential to replace the traditional, simulation-driven approach to
 science. Instead of relying on models that are carefully crafted
 from first principles, can data collections of sufficient size
 be processed to provide the correct answers instead?
+In short: this concern is unfounded. As we'll show in the next chapters,
+it is crucial to bring together both worlds: _classical numerical techniques_
+and _deep learning_.
 
-Very clear advantages of data-driven approaches would lead
+One central reason for the importance of this combination is
-to a "yes" here ... but that's not where we stand as of this writing.
+that DL approaches are simply not powerful enough by themselves.
-Given the current state of the art, these clear breakthroughs
+Given the current state of the art, the clear breakthroughs of DL
-are outstanding, the proposed techniques are novel,
+in physical applications are outstanding, the proposed techniques are novel,
 sometimes difficult to apply, and
-significant difficulties combing physics and DL persist.
+significant practical difficulties combing physics and DL persist.
 Also, many fundamental theoretical questions remain unaddressed, most importantly
 regarding data efficienty and generalization.
 
@@ -47,22 +59,23 @@ been developed to solve fundamental model equations such
 as the Navier-Stokes, Maxwell’s, or Schroedinger’s equations.
 Seemingly trivial changes to the discretization can determine
 whether key phenomena are visible in the solutions or not.
+Rather than discarding the powerful methods that have been
+carefully developed in the field of numerical mathematics, it 
+is highly beneficial for DL to use them as much as possible.
 
-```{admonition} Goal of this document
+```{admonition} Goals of this document
 :class: tip
-Thus, a key aspect that we want to address in the following in the following is:
+Thus, the key aspects that we want to address in the following are:
 - explain how to use DL,
-- how to combine it with existing knowledge of physics and simulations,
+- and how to combine it with existing knowledge of physics and simulations,
-- **without throwing away** all existing numerical knowledge and techniques!
+- **without throwing away** all existing numerical knowledge and techniques.
 ```
 
-Rather, we want to build on all the neat techniques that we have
+Thus, we want to build on all the powerful techniques that we have
-at our disposal, and use them as
+at our disposal, and use them wherever we can.
-much as possible. I.e., our goal is to _reconcile_ the data-centered
+I.e., our goal is to _reconcile_ the data-centered
 viewpoint and the physical simuation viewpoint.
 
-Also interesting: from a math standpoint ...
-''just'' non-linear optimization ...
 
 
 ## Categorization
@@ -124,8 +137,9 @@ each of the different techniques is particularly useful.
 A brief look at our _Notation_ won't hurt in both cases, though!
 ```
 
+---
 
-## A brief history of PBDL in the context of Fluids
+<!-- ## A brief history of PBDL in the context of Fluids
 
 First:
 
@@ -135,7 +149,7 @@ Chu, descriptors, early but not used
 
 Ling et al. isotropic turb, small FC, unused?
 
-PINNs ... and more ...
+PINNs ... and more ... -->
 
 
 ## Deep Learning and Neural Networks
@@ -160,4 +174,5 @@ we only deal with _regression_ problems in the following.
 
 maximum likelihood estimation
 
+Also interesting: from a math standpoint ''just'' non-linear optimization ...
 

references.bib

@@ -787,9 +787,60 @@
   publisher={Elsevier}
 }
 
+@book{stocker2014climate,
+  title={Climate change 2013: the physical science basis: Working Group I contribution to the Fifth assessment report of the Intergovernmental Panel on Climate Change},
+  author={Stocker, Thomas},
+  year={2014},
+  publisher={Cambridge university press}
+}
 
+@article{maingi2019fesreport,
+  title={Summary of the FESAC transformative enabling capabilities panel report},
+  author={Maingi, Rajesh and Lumsdaine, Arnold and Allain, Jean Paul and Chacon, Luis and Gourlay, SA and others},
+  journal={Fusion Science and Technology},
+  volume={75},
+  number={3},
+  pages={167--177},
+  year={2019},
+  publisher={Taylor Francis}
+}
 
+@article{o2016scalable,
+  title={Scalable quantum simulation of molecular energies},
+  author={O’Malley, Peter JJ and Babbush, Ryan and Kivlichan, Ian D and Romero, Jonathan and McClean, Jarrod R and Barends, Rami and Kelly, Julian and Roushan, Pedram and Tranter, Andrew and Ding, Nan and others},
+  journal={Physical Review X},
+  volume={6},
+  number={3},
+  pages={031007},
+  year={2016},
+  publisher={APS}
+}
 
+@inproceedings{krizhevsky2012,
+  title =    {ImageNet Classification with Deep Convolutional Neural Networks},
+  author =   {Alex Krizhevsky and Sutskever, Ilya and Hinton, Geoffrey E},
+  booktitle = {Advances in Neural Information Processing Systems},
+  year =2012,
+}
 
+@article{radford2019language,
+  title={Language models are unsupervised multitask learners},
+  author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},
+  journal={OpenAI blog},
+  volume={1},
+  number={8},
+  pages={9},
+  year={2019}
+}
 
+@article{alquraishi2019alphafold,
+  title={AlphaFold at CASP13},
+  author={AlQuraishi, Mohammed},
+  journal={Bioinformatics},
+  volume={35},
+  number={22},
+  pages={4862--4865},
+  year={2019},
+  publisher={Oxford University Press}
+}