daviddoji/pbdl-book
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

PG code discussion

Browse Source
This commit is contained in:
NT
2021-03-21 16:21:30 +08:00
parent 9a3f1cc46a
commit fc6547f78f
4 changed files with 327 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files

481
physgrad-comparison.ipynb
View File

File diff suppressed because one or more lines are too long

2
physgrad-nn.md
View File

@@ -1,4 +1,4 @@
Physical Gradients Physical Gradients and NNs
======================= =======================
Re-cap? Re-cap?

6
physgrad.md
View File

@@ -340,10 +340,12 @@ Even when the Jacobian is singular (because the function is not injective, chaot
The update obtained with a regular gradient descent method has surprising shortcomings. The update obtained with a regular gradient descent method has surprising shortcomings.
The physical gradient instead allows us to more accurately backpropagate through nonlinear functions, provided that we have access to good inverse functions. The physical gradient instead allows us to more accurately backpropagate through nonlinear functions, provided that we have access to good inverse functions.
Before moving on to including PGs in NN training processes, the next example will illustrate ... Before moving on to including PGs in NN training processes, the next example will illustrate the differences between these approaches with a practical example.
**todo, integrate comments below?**
**TODO, sometime, integrate comments below?**
Old Note: Old Note:
The inverse function to a simulator is typically the time-reversed physical process. The inverse function to a simulator is typically the time-reversed physical process.

2
supervised-airfoils.ipynb
View File

@@ -269,7 +269,7 @@
" if not transposed:\n", " if not transposed:\n",
" block.add_module('%s_conv' % name, nn.Conv2d(in_c, out_c, kernel_size=size, stride=2, padding=pad, bias=True))\n", " block.add_module('%s_conv' % name, nn.Conv2d(in_c, out_c, kernel_size=size, stride=2, padding=pad, bias=True))\n",
" else:\n", " else:\n",
" block.add_module('%s_upsam' % name, nn.Upsample(scale_factor=2))\n", " block.add_module('%s_upsam' % name, nn.Upsample(scale_factor=2, mode='bilinear'))\n",
" # reduce kernel size by one for the upsampling (ie decoder part)\n", " # reduce kernel size by one for the upsampling (ie decoder part)\n",
" block.add_module('%s_tconv' % name, nn.Conv2d(in_c, out_c, kernel_size=(size-1), stride=1, padding=pad, bias=True))\n", " block.add_module('%s_tconv' % name, nn.Conv2d(in_c, out_c, kernel_size=(size-1), stride=1, padding=pad, bias=True))\n",
"\n", "\n",