updated supervised airfoils notebook
This commit is contained in:
N_T committed
1 parent
285bff8b95
commit
960887d527
1 file changed
+233
-379
+233
-379
@@ -8,7 +8,7 @@
|
||||
"source": [
|
||||
"# Supervised training for RANS flows around airfoils\n",
|
||||
"\n",
|
||||
"## Overview \n",
|
||||
"## Overview\n",
|
||||
"\n",
|
||||
"For this example of supervised training\n",
|
||||
"we have a turbulent airflow around wing profiles, and we'd like to know the average motion\n",
|
||||
@@ -33,7 +33,7 @@
|
||||
"## Formulation\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"With the supervised formulation from {doc}`supervised`, our learning task is pretty straight-forward, and can be written as \n",
|
||||
"With the supervised formulation from {doc}`supervised`, our learning task is pretty straight-forward, and can be written as\n",
|
||||
"\n",
|
||||
"$$\\begin{aligned}\n",
|
||||
"\\text{arg min}_{\\theta} \\sum_i ( f(x_i ; \\theta)-y^*_i )^2 ,\n",
|
||||
@@ -43,12 +43,12 @@
|
||||
"and the index $i$ evaluates the difference across all discretization points in our data sets.\n",
|
||||
"\n",
|
||||
"The goal is to infer velocity $\\mathbf{u} = u_x,u_y$ and a pressure field $p$ in a computational domain $\\Omega$\n",
|
||||
"around the airfoil in the center of $\\Omega$. \n",
|
||||
"around the airfoil in the center of $\\Omega$.\n",
|
||||
"$u_x,u_y$ and $p$ each have a dimension of $128^2$.\n",
|
||||
"As inputs we have the Reynolds number $\\text{Re} \\in \\mathbb{R}$, the angle of attack\n",
|
||||
"$\\alpha \\in \\mathbb{R}$, and the airfoil shape $\\mathbf{s}$ encoded as a rasterized grid with $128^2$.\n",
|
||||
" $\\text{Re}$ and $\\alpha$ are provided in terms of the freestream flow velocity $\\mathbf{f}$, the x and y components of which are represented as constant fields of the same size, and contain zeros in the airfoil region.\n",
|
||||
"Thus, put together, both input and output have the same dimensions: $x,y^* \\in \\mathbb{R}^{3\\times128\\times128}$. The inputs contain \n",
|
||||
"Thus, put together, both input and output have the same dimensions: $x,y^* \\in \\mathbb{R}^{3\\times128\\times128}$. The inputs contain\n",
|
||||
"$[f_x,f_y,\\text{mask}]$, while the outputs store the channels $[p,u_x,u_y]$.\n",
|
||||
"This is exactly what we'll specify as input and output dimensions for the NN below.\n",
|
||||
"\n",
|
||||
@@ -63,7 +63,7 @@
|
||||
"source": [
|
||||
"## Code coming up...\n",
|
||||
"\n",
|
||||
"Let's get started with the implementation. Note that we'll skip the data generation process here. The code below is adapted from {cite}`thuerey2020dfp` and [this codebase](https://github.com/thunil/Deep-Flow-Prediction), which you can check out for details. Here, we'll simply download a small set of training data generated with a Spalart-Almaras RANS simulation in [OpenFOAM](https://openfoam.org/)."
|
||||
"Let's get started with the implementation. Note that we'll skip the data generation process here. The code below is adapted from {cite}`thuerey2020dfp` and [this codebase](https://github.com/thunil/Deep-Flow-Prediction), which you can check out for details. Below, we'll simply use a small set of training data generated with a Spalart-Almaras RANS simulation in [OpenFOAM](https://openfoam.org/). First, let's import the required module, and install the dataloader from git."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -74,63 +74,60 @@
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "JwZudtWauiGa",
|
||||
"outputId": "ac464e27-1aad-4a64-d748-6bb1b6b566a5"
|
||||
"outputId": "60e43840-2888-46aa-a750-8453b3b606aa"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os, sys, random\n",
|
||||
"import numpy as np\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"import torch\n",
|
||||
"import torch.nn as nn\n",
|
||||
"import torch.optim as optim\n",
|
||||
"\n",
|
||||
"!pip install --upgrade --quiet git+https://github.com/tum-pbs/pbdl-dataset@develop\n",
|
||||
"from pbdl.torch.loader import Dataloader"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cghhfbQjiPIB"
|
||||
},
|
||||
"source": [
|
||||
"The next cell will download the training data from HuggingFace, which can take a moment... The PBDL dataloader call below directly splits it into 320 samples for training, and 80 samples for validation. These validation samples are using the same airfoil shapes as the training samples, but different conditions (later on we'll download new shapes for testing)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "OmYUCtpeiPIB",
|
||||
"outputId": "f5427adb-d15b-4470-ac18-be4859831a53"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Torch version 1.11.0+cu113\n",
|
||||
"Downloading training data (300MB), this can take a few minutes the first time...\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loaded data, 320 training, 80 validation samples\n",
|
||||
"Size of the inputs array: (320, 3, 128, 128)\n"
|
||||
"\u001b[93m\u001b[1mWarning:\u001b[22m `airfoils` is stored in single-file format. The download might take some time.\u001b[0m\n",
|
||||
"\u001b[96m\u001b[1mSuccess:\u001b[22m Loaded airfoils with 400 simulations and 1 samples each.\u001b[0m\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import os.path, random\n",
|
||||
"import torch\n",
|
||||
"from torch.utils.data import Dataset\n",
|
||||
"print(\"Torch version {}\".format(torch.__version__))\n",
|
||||
"BATCH_SIZE = 10\n",
|
||||
"\n",
|
||||
"# get training data\n",
|
||||
"dir = \"./\"\n",
|
||||
"if True:\n",
|
||||
" # download\n",
|
||||
" if not os.path.isfile('data-airfoils.npz'):\n",
|
||||
" import requests\n",
|
||||
" print(\"Downloading training data (300MB), this can take a few minutes the first time...\")\n",
|
||||
" with open(\"data-airfoils.npz\", 'wb') as datafile:\n",
|
||||
" resp = requests.get('https://dataserv.ub.tum.de/s/m1615239/download?path=%2F&files=dfp-data-400.npz', verify=False)\n",
|
||||
" datafile.write(resp.content)\n",
|
||||
"else: \n",
|
||||
" # alternative: load from google drive (upload there beforehand):\n",
|
||||
" from google.colab import drive\n",
|
||||
" drive.mount('/content/gdrive')\n",
|
||||
" dir = \"./gdrive/My Drive/\"\n",
|
||||
"\n",
|
||||
"npfile=np.load(dir+'data-airfoils.npz')\n",
|
||||
" \n",
|
||||
"print(\"Loaded data, {} training, {} validation samples\".format(len(npfile[\"inputs\"]),len(npfile[\"vinputs\"])))\n",
|
||||
"\n",
|
||||
"print(\"Size of the inputs array: \"+format(npfile[\"inputs\"].shape))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "u488C6ybnMCN"
|
||||
},
|
||||
"source": [
|
||||
"If you run this notebook in colab, the `else` statement above (which is deactivated by default) might be interesting for you: instead of downloading the training data anew every time, you can manually download it once and store it in your google drive. We assume it's stored in the root directory as `data-airfoils.npz`. Afterwards, you can use the code above to load the file from your google drive, which is typically much faster. This is highly recommended if you want to experiment more extensively via colab."
|
||||
"loader_train, loader_val = Dataloader.new_split(\n",
|
||||
" [320, 80],\n",
|
||||
" \"airfoils\",\n",
|
||||
" batch_size=BATCH_SIZE, normalize_data=None,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -141,119 +138,59 @@
|
||||
"source": [
|
||||
"## RANS training data\n",
|
||||
"\n",
|
||||
"Now we have some training data. In general it's very important to understand the data we're working with as much as possible (for any ML task the _garbage-in-gargabe-out_ principle definitely holds). We should at least understand the data in terms of dimensions and rough statistics, but ideally also in terms of content. Otherwise we'll have a very hard time interpreting the results of a training run. And despite all the DL magic: if you can't make out any patterns in your data, NNs surely won't find any useful ones.\n",
|
||||
"Now we have the training and validation data. In general it's very important to understand the data we're working with as much as possible (for any ML task the _garbage-in-gargabe-out_ principle definitely holds). We should at least understand the data in terms of dimensions and rough statistics, but ideally also in terms of content. Otherwise we'll have a very hard time interpreting the results of a training run. And despite all the DL magic: if you can't make out any patterns in your data, NNs surely won't find any useful ones.\n",
|
||||
"\n",
|
||||
"Hence, let's look at one of the training samples... The following is just some helper code to show images side by side."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 353
|
||||
"height": 505
|
||||
},
|
||||
"id": "6y6YGxMeNECD",
|
||||
"outputId": "ac8e0746-a975-4dc9-de80-cc8202b36f6e"
|
||||
"outputId": "5ef5f729-de46-4312-9f97-72bdb3da0ea4"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAo0AAAE8CAYAAACl04sAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydebwsR1n3v0/3zDnnJvdmT1iSkLALYROQRUDzsgiE7VV22UFEkU1ANhEREMSVRRBQMZCwia9okCAISCJb2AxgwhYDITeQkIVs995z78z08/5R1TM1Pd0z3T0zZ6bPPN/zmc+Z6aW6uru66tfP81SVqCqGYRiGYRiGMY5o0RkwDMMwDMMwlh8TjYZhGIZhGMZETDQahmEYhmEYEzHRaBiGYRiGYUzERKNhGIZhGIYxERONhmEYhmEYxkRmKhpF5Aki8slZprldEJGTRWT3ovMxT0TkRBFREWktOi9lEJFnicibgt+/KiIXi8j1IvLzi8zbNIjIDhH5qIhcIyIfXnR+lh0RebWInF6w7rki8sY5HHNsXTmL+sI/i7eYJo1VRkROFZHXLTofWUTkhyJy/xr7rYvI+SJyo3nka974/H9HRI7e4uPOrByIyGdF5DdmkdaimCgaReR0EfmJiFwrIt8bd8Kq+j5V/ZXZZjE3T40SJ01kXEMabFOr8ip5/LmKbBFZA14J/Fmw+M+B56jqTlX973kdewt4FHAD4EhVffSiM9Nw/hZ4gogcM8tEs3XltAJvno2RiBwlIp8XkStF5GoR+aKI3GtOx5rpeWyHRnqG/CZwtqr+ZNEZqYOq7gfeDbysbhqTnjMReaqIfK5u+vMk294uSgeVsTS+AThRVQ8BHg68TkTuMt9sLQ8iEi86D8ZceATwHVW9JFh2AnBe3sYNe0E5AfieqnbzVjbsXBaKqm4CHweevOi8LJDrgacDRwOHA28EPmrlqHH8FnDaojMxJe8HniIi64vOyMqiqqU/wK2BnwCPKVj/VOBzwW/FFdTvA1cDbwMk2PbzwF8D1wDfAe4X7PtD4P7B71cDp/vvP/JpX+8/9wRuAZzl07oC+NCY8/gwcKnf9mzgpGDdqcDfAGcCe4D7AzcG/h9wOfAD4Hlj0j4FOB+4DrgEeLFffjKwG3gR8FN/HZ8W7Hco8F5/jItwVrDIr7sIuIv//gR/7if5388A/iW4Rv/o07kOJ4DuOiavbwYuBq4Fvgbcxy9/EHAA6Pjr+42cfU8DEmCf3+YlwIk+b0/x9+gK4PeDfSLcW+L/Alf6vB6Rk/bBPt0kuMc3BtaBNwE/9p83AeuZ6/sKf9wfAk8Yc+7vBl7pv6/7Y6i/5/8blMGXAt8E9gMt4B7AF3Dl+RvAyZl7+Pf+3l4CvA6IC46fXqtWsOyzwG/4738D/L9g3RuBT+OfnzHn9UeZe/cMBs/aX/nr/jp/zn/u79NlwDuAHUE6DwXO9ef5BeAOY455KvC64PfJwG7//ffC8/DL3gK8uUR9k16jp+HK6c9w9ckv+HtyNfDXwfY3Bz7jz/EK4H3AYcH6l/r7ch3wXXx9w3Dd0gY+gHve14Jn7j9L1pFnAY/03+/l8/8Q//t+wLnZuhJXB6Vl73rgsUyoLzLH/GOgB2z6/f96Uv3r1z8d+La/rp8ATihxfhHwMJ/2MQXbjKvL+tc6+xxMOI/nARf6+/pn06SXk997M3imLwaeGpTrtwEf82XmHODmk+rPMnUxrm55Ma4cXwN8CNgo8/wRtI3A3YCv+jxcBvxlwTneBFenhvXNqbjn/j98Hs8qKgNk2vbgvtwCWPN5fa5fHuPqm1cVpNXPf87z97E0nWD9N4FfDX5/H/jlgrTvBnzRX7ef4PRF+hyPPGeZfW/jy0rPr7+6ZDn4OX8Nr8LVK7n6yG/7WZwh7sv+nv0rQRuIM8yd5/P/WeA2fnlee5ungyLc83YRrt54L3Bonfq08BxKVoRvB/b6A34d2FmmYPnt/w04DFdoLwceFGzbBX4XV1E/FvfwHFGiYKUnHz4AHwB+31+0DeDeY87n6cAuBiLk3MyDdA2uwo+Ag3AVwqtwD8fNcJXXAwvS/gkD8XU4cGf//WR/vq/x53uKv6aH+/Xv9QVolz+/7wHPCNa9yH9/F050/Xaw7neDa7Tp045xhfNLY67DE4EjcRXsi3BCeiN7vcfsn71H6X35W2AHcEec2EoL/vOBLwHH+Wv/TuADBWmfjBcewbLX+P2PwVk9vgC8NnN9/9Kn/cu4yuHWBel/BXh0XiWYOb9zgeP9+RyLEySn+LLxAP/7aL/9R/w5Hezz+GXgWQXHT69VkWg8yJeBpwL3wTWWx5V8XofuHYNn7bn+Xu/ACcgzgCNwZe6jwBv89j+Pq3Du7svRU/y1WC843qkUi8Yb+ftwmP/d8mnfpcR5pNfoHbhn+ldw5ftf/PU91qf1y377W/h7su7Lx9nAm/y6W+MqyhsHad88vF7+unzMn08c5OPOwFUlr/1rgLf676/APatvDNa9eUxdGZa9kxlTX+Qct192MmkW1b+PAC7ANZQtXEPzhQnn9k3cC4kCfztmu3F12aspEHkTzuM/cWX1Jj6936ibXibtE3BC4PH+Oh8J3Cko11fihEgL9xLywWDfSfVnYV2Me56+jHsZPgIn3n+rzPPHsGj8IvAk/30ncI+C83wIcF7Oc3sd8Eu4Z+bNZIRhpg7JFY3+++1wIuQ2uHb4SxS/MPfzn72HwGOAc4J1d/T3YC1YdgYFhhvgLrgX+5YvC98GXlD0nJU8z8JygKvrL8YJsZa/d1cAtx3znF7ir9fBuJfT9NxvhasrH4Ariy/BPaNrBdftREbbkKf7fW7my8M/A6dlti9VnxZeo3ErMycb497IXgm0y1xwn8F7B7//EXhZsO2PGX7z/TKDB2Bcwcq7WO/FCapSDWuw32E+rVSNnwq8N1h/d+BHmX1eDvxDQXo/Ap4FHJJZfjKjb3o/xRXwGFcZ3zZY9yzgs/77M4Az/PdvA7/BoNBexECYvhr4VJDGbYF9Fa7Fz4A7Zq/3mO2LCvFxwbIvA48L8h5ak2+Es4i1ctI+mVHR+L/AKcHvBwI/DLbvAgdnytsfFOT9+/gGNFNes6Lx6cHvl+IfwGDZJ3CV+g1wAjm01j2eAgtVQRn+LEED58veVf4eP77CfRy6d7hn7UfBb8FVTuHb8j2BH/jvf4MX48H671L8dn8qBaLR//448Ez//aHA+SXPI71GxwbLriSwEOAq3RcU7P9/gf/232+Be97uT6b+8tfrDJyl5S1krLnALYFeyTzfD/im//7vuGf1S/73WcCvBfdkkmjMrS8KjjtUdoI0i+rfj+OFnP8d4UTpCRPOb8OX66cUrJ9Ul2XLZnqPJ4nGBwW/nw18um56mbRfDnxkTLn+u+D3KbiQlrL1Z2FdjKtbnhj8/lPgHWWeP4ZF49k478JRE+7bE8gYEPz5hSJ4J87KdnzO/kPltaDMvsjn82fALcfkpZ//7D305au/P84b8vbM/u+jwIqZc6wXhPc3m+eS51lYDnDGrv/KbP9O4A8L0v8s8CeZcnEA99z8AfCPwboIJzBPLrhuJzLahnwaeHbw+9b4NpYp69P0U7r3tKr2VPVzOCvRb5fdD/f2lbIXVzBTLlGfU89FuDevOrwE1xh+WUTOE5Gn520kIrGI/ImI/K+IXIu7EQBHBZtdHHw/AbixDwC/WkSuxlkQblCQj0fiCtVFInKWiNwzWHelDseZpdfjKNybxUXBuotwyh9cY3Mf3+stxlX+9xKRE3GuoHOD/bLXe6Mo9khEXiwi3/a9bK/2aR2Vt21Fiu75CcBHguv4bVwlVXQts9yY0WsUlpefqeqeMetDfoazhEwiWxYenSkL98aJ3xNw9/Anwbp34t7g8GXyev+5T4njoqrn4Kzagrvn0xCex9F4C3qQ13/3y9PzfFHmPI/HPQdPCM7j4yWP/R6cVQb/v2pc1WXB9305v3cCiMgNROSDInKJf7ZPx5dnVb0A14C8Gvip3y4sG/cA7oCr0MM6CVw5uaZkXr8I3EpEbgDcCfcye7yIHIWzVJxdMh0ori+qMO5ZfHNwf6/ClbNjGYOqbqrqB4CXicgdczaZVJfVJSy/07QTWY7HvYwWUdh+lag/J9XF4+5N7vOXk79n4CxU3xGRr4jIQwvOo6i+619XVb0eVw7qXtv3+Lyfqarfr5OAuhjiDwFPFJEI94KSrS924VypI4jIrUTk30TkUl8HvJ75t2l3z9yrJwA3HJNWtiy3fR6H2jdVTfy2VZ6dvDayxXAbW6o+LaLOkDstXOzQLDhWRCT4fROc9RGcJeSgYF14E7KVOqp6qao+U1VvjHuzfXtBL6lfx7lm7o97yE/0y8N8hOlfjLPAHBZ8dqnqKXknpKpfUdVH4MTCv1Cusb8C9zZwQrDsJri3jLTB24tzL56tqtfiCvFv4t6KkhLHGMILl5fg3AGHq+phuIYxvQ4j1ziHMtuEXAw8OHMtN3S4M8q4tH/M6DX6cfD7cBE5eMz6kG/iKttJZMvCaZn8H6yqf+LX7ce98afrDlHVkwBU9SR1vbJ3qup/4co3FJdxROR3cG6jH+Pu1TSE53EFrnI4KcjroaqaVhYXA3+cOc+DVPUD6nr9pufxYL/9uGcV3HNwBxG5Hc7S+L4pz6WI1+PO8/bqOu49keC5VtX3q+q9cWVIcXGiKZ/EuRA/7QVfyG1w8asTUdW9uHCW5wP/o6oHcGEUL8TFyl5R58TKHLri9hfjQifCe7xDVb9Qcv82zgWWZWxdxuSyUnQex2fSK9NOjEsv5WJqtGcl6s9pKHz+shuq6vdV9fG49uaNwD9l6sCUbwI3zTEg9K+riOzEucrz6syh6ywieaLo7bhwiAeKyL3HnN+ke/YenPC6H7BXVb+YWT/uefwbXP+IW/o64BVUuyd1nqOzMvdqp6qOM6xly3IH99wMtW9eGx3P4NnJ5q1Line truncated
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAHTCAYAAAAZCb9mAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAADpGUlEQVR4nOydd5zU1PrGn2RmG7vsLh2lF5UqKAg2wIKiImDhYqfYC1eF608sVxG86lUUURSxAmJD0Wu/IioqdkXh2sBGUaRK3V12Z2fy/v7IJJNkTtpMZnaXfb9+RnaSk5OTTMqTJ+95j0REBIZhGIZhGIZhAkWu6QYwDMMwDMMwzN4IC22GYRiGYRiGyQAstBmGYRiGYRgmA7DQZhiGYRiGYZgMwEKbYRiGYRiGYTIAC22GYRiGYRiGyQAstBmGYRiGYRgmA7DQZhiGYRiGYZgMwEKbYRiGYRiGYTIAC+06yC233AJJkmq6GfWaNWvWQJIk3H333TXdlDrHUUcdhR49etR0M7LG888/j8aNG6OsrEyfFo1Gce2116JNmzaQZRmnnHJKzTWwFrFp0yaMHDkSTZo0gSRJmDFjRk03iTEwd+5cSJKEr776yrHcddddh/79+2epVQxTu2GhXQN8//33+Nvf/oaOHTuiQYMGaNq0KQYOHIjXXnutppuWEp988gluueUW7Nixo6abwmSQ22+/HS+//LKnsn/++SduueUWLF++PKNtcuKZZ56pcaEWi8UwefJk/P3vf0dRUZE+/YknnsC0adMwcuRIzJs3DxMmTKjBVtYeJkyYgEWLFuH666/H/PnzccIJJ9R0k5gUuPrqq7FixQq8+uqrNd0UhqlxWGjXAGvXrsXu3bsxZswY3HfffbjpppsAAMOHD8cjjzziuvw///lP7NmzJ9PN9Mwnn3yCKVOmsNDey/ErtKdMmVLvhfZrr72GVatW4eKLLzZNf++999CqVSvce++9OO+88zBo0KAaamHt4r333sOIESNwzTXX4Nxzz0WXLl1quklMCrRs2RIjRozgN34MAyBc0w2oj5x00kk46aSTTNPGjx+PPn36YPr06Uk3ZSvhcBjhMP90IogIlZWVKCgoqOmmMAzmzJmDI444Aq1atTJN37x5M0pLS12Xj0ajUBQFubm5GWph7cLrfikvL0dhYWHmG8SkzKhRo/C3v/0Nv/32Gzp27FjTzWGYGoMd7VpCKBRCmzZtPLnCohhtSZIwfvx4vPzyy+jRowfy8vLQvXt3vPXWW8JlV65ciVGjRqG4uBhNmjTBVVddhcrKSr2cFoM8d+7cpPVLkoRbbrlFr+///u//AAAdOnSAJEmQJAlr1qwBACxevBhHHnkkSktLUVRUhAMOOAA33HCD6zbOmTMHxxxzDJo3b468vDx069YNDz30UFK59u3b4+STT8aiRYvQt29fFBQU4OGHHwYA7NixA1dffTXatGmDvLw8dO7cGXfeeScURXFd/1dffYUhQ4agadOmKCgoQIcOHXD++ecLyz7yyCPo1KkT8vLycMghh+DLL79MKvPee+9hwIABKCwsRGlpKUaMGIEff/xRn/+///0PkiSZXrUuW7YMkiTh4IMPNtV14oknmuIftX3w0UcfoV+/fsjPz0fHjh3x5JNPum4nANx99904/PDD0aRJExQUFKBPnz5YuHChqYwkSSgvL8e8efP033js2LHC+t5//30ccsghAIBx48bp5a3H0g8//ICjjz4aDRo0QKtWrXDXXXcl1VVVVYXJkyejc+fOyMvLQ5s2bXDttdeiqqrKcZuOOuoovPHGG1i7dq2+/vbt2+vzN2/ejAsuuAAtWrRAfn4+evXqhXnz5pnqMMbh33vvvWjXrh0KCgowaNAgfPfdd47rB4DKykq89dZbGDx4cFKdS5Yswffff6+37f333zetb8aMGfox9cMPPwAAVq5ciZEjR6Jx48bIz89H3759ha/m0znutRhc7fzVeP/99/V2AsCPP/6IgoICjB492lTuo48+QigUwqRJk1zXJVovEeHBBx/U94tx3gcffIDLL78czZs3R+vWrfVl//vf/+rnVsOGDTF06FB8//33Sevwuv9EjB071nT8aFivxYMGDUKvXr2EdRxwwAEYMmSIp/WJ1vHTTz/h3HPPRUlJCZo1a4abbroJRITff/8dI0aMQHFxMVq2bIl77rnHtHwkEsHNN9+MPn36oKSkBIWFhRgwYACWLFmStK7nnnsOffr0QcOGDVFcXIyePXvivvvuc2zf9u3b0a9fP7Ru3RqrVq3Sp2vH/SuvvOJ7mxlmr4KYGqOsrIy2bNlCv/zyC02fPp1CoRCdffbZrstNnjyZrD8dAOrVqxfts88+dOutt9KMGTOoY8eO1KBBA9q6dWvSsj179qRhw4bRAw88QOeeey4BoPPOO08vt3r1agJAc+bMSVo/AJo8eTIREa1YsYLOOussAkD33nsvzZ8/n+bPn09lZWX03XffUW5uLvXt25fuu+8+mj17Nl1zzTU0cOBA12085JBDaOzYsXTvvffSzJkz6fjjjycA9MADD5jKtWvXjjp37kyNGjWi6667jmbPnk1Lliyh8vJyOvDAA6lJkyZ0ww030OzZs2n06NEkSRJdddVVjuvetGkTNWrUiPbff3+aNm0aPfroo3TjjTdS165dk/bPQQcdRJ07d6Y777yT7rrrLmratCm1bt2aIpGIXnbx4sUUDodp//33p7vuuoumTJlCTZs2pUaNGtHq1auJiCgWi1FpaSn94x//0Je79957SZZlkmWZdu7cqZcrLi6ma665xrQPDjjgAGrRogXdcMMN9MADD9DBBx9MkiTRd99957qvW7duTZdffjk98MADNH36dOrXrx8BoNdff10vM3/+fMrLy6MBAwbov/Enn3wirG/jxo00depUAkAXX3yxXv7XX38lIqJBgwbRvvvuS23atKGrrrqKZs2aRccccwwBoDfffFOvJxaL0fHHH08NGjSgq6++mh5++GEaP348hcNhGjFihOM2vf3229S7d29q2rSpvv7//Oc/RERUUVFBXbt2pZycHJowYQLdf//9NGDAAAJAM2bM0OvQfuOePXtS+/bt6c4776QpU6ZQ48aNqVmzZrRx40bHNnz00UcEgF599VV9WllZGc2fP5+6dOlCrVu31tu2ceNGfX3dunWjjh070r///W+69957ae3atfTdd99RSUkJdevWje6880564IEHaODAgSRJEr300kt6/ekc90REc+bMIQD6camxZMkSAkBLlizRp02bNo0A0CuvvKJvW6dOnahbt25UWVnpui4jv/76K82fP58A0HHHHafvF2ObunXrRoMGDaKZM2fSv//9byIievLJJ0mSJDrhhBNo5syZdOedd1L79u2ptLTUtA1e958dY8aMoXbt2iVNt16LH330UQJA3377rancF198QQDoySef9LVfjOvo3bs3nXXWWTRr1iwaOnQoAaDp06fTAQccQJdddhnNmjWLjjjiCAJAH3zwgb78li1baJ999qGJEyfSQw89RHfddRcdcMABlJOTQ998841e7u233yYAdOyxx9KDDz5IDz74II0fP57+9re/6WW03+LLL7/U6+7duze1bduWfvnll6S2d+7cmU4//XTf28wwexMstGuQSy65hAAQAJJlmUaOHEnbtm1zXc5OaOfm5pouditWrCAANHPmzKRlhw8fblr+8ssvJwC0YsUKIvIutIkSN1zrzfnee+8lALRlyxbXbbJSUVGRNG3IkCHUsWNH07R27doRAHrrrbdM02+99VYqLCykn376yTT9uuuuo1AoROvWrbNd93/+8x/TzUSEtn+aNGli+s1eeeUVAkCvvfaaPq13797UvHlz+uuvv/RpK1asIFmWafTo0fq0oUOHUr9+/fTvp512Gp122mkUCoXov//9LxERff311yZxY9wHH374oT5t8+bNlJeXZxLudlj3dSQSoR49etAxxxxjml5YWEhjxoxxrY+I6Msvv7Q9fgYNGpQkOqqqqqhly5amm/L8+fNJlmVaunSpafnZs2cTAPr4448d2zB06FChOJoxYwYBoKeeekqfFolE6LDDDqOioiLatWsXESV+44KCAvrjjz/0sp9//jkBoAkTJjiu/7HHHhOKLm0fdO/e3TRNW19xcTFt3rzZNO/YY4+lnj17mgSsoih0+OGH03777adPS+e4J/IntGOxGB155JHUokUL2rp1K11xxRUUDocdzxs3ANAVV1whbNORRx5J0WhUn757924qLS2liy66yFR+48aNVFJSYprudf/Z4VVo79ixg/Lz82nSpEmmcldeeSUVFhZSWVmZ67rs1nHxxRfr06LRKLVu3ZokSdIfOoiItm/fTgUFBabzNBqNUlVVlanO7du3U4sWLej888/Xp1111VVUXFxs2sdWjEJ7w4YN1L17d+rYsSOtWbNGWP744483GRQMUx/h0JEa5Oqrr8bixYsxb948nHjiiYjFYohEIinXN3jwYHTq1En/fuCBB6K4uBi//fZbUtkrrrjC9P3vf/87AODNN99Mef1WtFjLV155xdNrayPGGOudO3di69atGDRoEH777Tfs3LnTVLZDhw5Jr2RfeOEFDBgwAI0aNcLWrVv1z+DBgxGLxfDhhx+6tvv1119HdXW1YzvPOOMMNGrUSP8+YMAAAND3+YYNG7B8+XKMHTsWjRs31ssdeOCBOO6440z7e8CAAfj6669RXl4OQH0Nf9JJJ6F3795YunQpAGDp0qWQJAlHHnmkqR3dunXT1w0AzZo1wwEHHCD87a0Y9/X27duxc+dOvS2ZoqioCOeee67+PTc3F/369TO194UXXkDXrl3RpUsX0294zDHHAIDw1bcX3nzzTbRs2RJnnXWWPi0nJwdXXnklysrK8MEHH5jKn3LKKaYY6379+qF///6u58pff/0FAKbjwwunn346mjVrpn/ftm0b3nvvPYwaNQq7d+/W98NLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1152x360 with 2 Axes>"
|
||||
"<Figure size 1600x500 with 2 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"needs_background": "light"
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import pylab\n",
|
||||
"def plot(a1, a2, mask=None, stats=False, bottom=\"NN Output\", top=\"Reference\", title=None):\n",
|
||||
" c = []\n",
|
||||
" if mask is not None: mask = np.asarray(mask)\n",
|
||||
" for i in range(3):\n",
|
||||
" a2i = np.asarray(a2[i])\n",
|
||||
" if mask is not None: a2i = a2i - mask*a2i # optionally mask out inner region\n",
|
||||
" b = np.flipud(np.concatenate((a2i, a1[i]), axis=1).transpose())\n",
|
||||
" min, mean, max = np.min(b), np.mean(b), np.max(b)\n",
|
||||
" if stats:\n",
|
||||
" print(\"Stats %d: \" % i + format([min, mean, max]))\n",
|
||||
" b -= min\n",
|
||||
" b /= max - min\n",
|
||||
" c.append(b)\n",
|
||||
" fig, axes = plt.subplots(1, 1, figsize=(16, 5))\n",
|
||||
" axes.set_xticks([]) ; axes.set_yticks([])\n",
|
||||
" im = axes.imshow(np.concatenate(c, axis=1), origin=\"upper\", cmap=\"magma\")\n",
|
||||
" fig.colorbar(im, ax=axes)\n",
|
||||
" axes.set_xlabel(\"p, ux, uy\")\n",
|
||||
" axes.set_ylabel(\"%s %s\" % (bottom, top))\n",
|
||||
" if title is not None: plt.title(title)\n",
|
||||
" plt.show()\n",
|
||||
"\n",
|
||||
"# helper to show three target channels: normalized, with colormap, side by side\n",
|
||||
"def showSbs(a1,a2, stats=False, bottom=\"NN Output\", top=\"Reference\", title=None): \n",
|
||||
" c=[]\n",
|
||||
" for i in range(3):\n",
|
||||
" b = np.flipud( np.concatenate((a2[i],a1[i]),axis=1).transpose())\n",
|
||||
" min, mean, max = np.min(b), np.mean(b), np.max(b); \n",
|
||||
" if stats: print(\"Stats %d: \"%i + format([min,mean,max]))\n",
|
||||
" b -= min; b /= (max-min)\n",
|
||||
" c.append(b)\n",
|
||||
" fig, axes = pylab.subplots(1, 1, figsize=(16, 5))\n",
|
||||
" axes.set_xticks([]); axes.set_yticks([]); \n",
|
||||
" im = axes.imshow(np.concatenate(c,axis=1), origin='upper', cmap='magma')\n",
|
||||
"\n",
|
||||
" pylab.colorbar(im); pylab.xlabel('p, ux, uy'); pylab.ylabel('%s %s'%(bottom,top))\n",
|
||||
" if title is not None: pylab.title(title)\n",
|
||||
"\n",
|
||||
"NUM=72\n",
|
||||
"showSbs(npfile[\"inputs\"][NUM],npfile[\"targets\"][NUM], stats=False, bottom=\"Target Output\", top=\"Inputs\", title=\"3 inputs are shown at the top (free-ux, free-uy, mask), with the 3 output channels (p,ux,uy) at the bottom\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "TVHhr8zCPUfN"
|
||||
},
|
||||
"source": [
|
||||
"Next, let's define a small helper class `DfpDataset` to organize inputs and targets. We'll transfer the corresponding data to the pytorch `DataLoader` class. \n",
|
||||
"\n",
|
||||
"We also set up some globals to control training parameters, maybe most importantly: the learning rate `LR`, i.e. $\\eta$ from the previous sections. When your training run doesn't converge this is the first parameter to experiment with.\n",
|
||||
"\n",
|
||||
"Here, we'll keep it relatively small throughout. (Using _learning rate decay_ would be better, i.e. potentially give an improved convergence, but is omitted here for clarity.) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "BGN4GqxkIueM",
|
||||
"outputId": "edafc7f9-9f88-4d6a-ebea-77500412ef1e"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training & validation batches: 32 , 8\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# some global training constants\n",
|
||||
"\n",
|
||||
"# number of training epochs\n",
|
||||
"EPOCHS = 100\n",
|
||||
"# batch size\n",
|
||||
"BATCH_SIZE = 10\n",
|
||||
"# learning rate\n",
|
||||
"LR = 0.00002\n",
|
||||
"\n",
|
||||
"class DfpDataset():\n",
|
||||
" def __init__(self, inputs,targets): \n",
|
||||
" self.inputs = inputs\n",
|
||||
" self.targets = targets\n",
|
||||
"\n",
|
||||
" def __len__(self):\n",
|
||||
" return len(self.inputs)\n",
|
||||
"\n",
|
||||
" def __getitem__(self, idx):\n",
|
||||
" return self.inputs[idx], self.targets[idx]\n",
|
||||
"\n",
|
||||
"tdata = DfpDataset(npfile[\"inputs\"],npfile[\"targets\"])\n",
|
||||
"vdata = DfpDataset(npfile[\"vinputs\"],npfile[\"vtargets\"])\n",
|
||||
"\n",
|
||||
"trainLoader = torch.utils.data.DataLoader(tdata, batch_size=BATCH_SIZE, shuffle=True , drop_last=True) \n",
|
||||
"valiLoader = torch.utils.data.DataLoader(vdata, batch_size=BATCH_SIZE, shuffle=False, drop_last=True) \n",
|
||||
"\n",
|
||||
"print(\"Training & validation batches: {} , {}\".format(len(trainLoader),len(valiLoader) ))"
|
||||
"inputs, targets = next(iter(loader_train))\n",
|
||||
"plot(inputs[0], targets[0], stats=False, bottom=\"Target Output\", top=\"Inputs\", title=\"3 inputs are shown at the top (free-ux, free-uy, mask),\\n with the 3 output channels (p,ux,uy) at the bottom\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -275,69 +212,70 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"id": "PVNjz4FjGhdY"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os, sys, random\n",
|
||||
"import numpy as np\n",
|
||||
"def blockUNet( in_c, out_c, name, size=4, pad=1, transposed=False, bn=True, activation=True, relu=True, dropout=0.0 ):\n",
|
||||
"\n",
|
||||
"import torch\n",
|
||||
"import torch.nn as nn\n",
|
||||
"import torch.optim as optim\n",
|
||||
"import torch.autograd \n",
|
||||
"import torch.utils.data \n",
|
||||
"\n",
|
||||
"def blockUNet(in_c, out_c, name, size=4, pad=1, transposed=False, bn=True, activation=True, relu=True, dropout=0. ):\n",
|
||||
" block = nn.Sequential()\n",
|
||||
"\n",
|
||||
" if not transposed:\n",
|
||||
" block.add_module('%s_conv' % name, nn.Conv2d(in_c, out_c, kernel_size=size, stride=2, padding=pad, bias=True))\n",
|
||||
" block.add_module(\n",
|
||||
" \"%s_conv\" % name,\n",
|
||||
" nn.Conv2d(in_c, out_c, kernel_size=size, stride=2, padding=pad, bias=True),\n",
|
||||
" )\n",
|
||||
" else:\n",
|
||||
" block.add_module('%s_upsam' % name, nn.Upsample(scale_factor=2, mode='bilinear'))\n",
|
||||
" block.add_module(\n",
|
||||
" \"%s_upsam\" % name, nn.Upsample(scale_factor=2, mode=\"bilinear\")\n",
|
||||
" )\n",
|
||||
" # reduce kernel size by one for the upsampling (ie decoder part)\n",
|
||||
" block.add_module('%s_tconv' % name, nn.Conv2d(in_c, out_c, kernel_size=(size-1), stride=1, padding=pad, bias=True))\n",
|
||||
" block.add_module(\n",
|
||||
" \"%s_tconv\" % name,\n",
|
||||
" nn.Conv2d( in_c, out_c, kernel_size=(size - 1), stride=1, padding=pad, bias=True ),\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" if bn:\n",
|
||||
" block.add_module('%s_bn' % name, nn.BatchNorm2d(out_c))\n",
|
||||
" if dropout>0.:\n",
|
||||
" block.add_module('%s_dropout' % name, nn.Dropout2d( dropout, inplace=True))\n",
|
||||
" block.add_module(\"%s_bn\" % name, nn.BatchNorm2d(out_c))\n",
|
||||
" if dropout > 0.0:\n",
|
||||
" block.add_module(\"%s_dropout\" % name, nn.Dropout2d(dropout, inplace=True))\n",
|
||||
"\n",
|
||||
" if activation:\n",
|
||||
" if relu:\n",
|
||||
" block.add_module('%s_relu' % name, nn.ReLU(inplace=True))\n",
|
||||
" block.add_module(\"%s_relu\" % name, nn.ReLU(inplace=True))\n",
|
||||
" else:\n",
|
||||
" block.add_module('%s_leakyrelu' % name, nn.LeakyReLU(0.2, inplace=True))\n",
|
||||
" block.add_module(\"%s_leakyrelu\" % name, nn.LeakyReLU(0.2, inplace=True))\n",
|
||||
"\n",
|
||||
" return block\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class DfpNet(nn.Module):\n",
|
||||
" def __init__(self, channelExponent=6, dropout=0.):\n",
|
||||
" def __init__(self, channelExponent=6, dropout=0.0):\n",
|
||||
" super(DfpNet, self).__init__()\n",
|
||||
" channels = int(2 ** channelExponent + 0.5)\n",
|
||||
" channels = int(2**channelExponent + 0.5)\n",
|
||||
"\n",
|
||||
" self.layer1 = blockUNet( 3, channels * 1, \"enc_layer1\", transposed=False, bn=True, relu=False, dropout=dropout, )\n",
|
||||
" self.layer2 = blockUNet( channels, channels * 2, \"enc_layer2\", transposed=False, bn=True, relu=False, dropout=dropout, )\n",
|
||||
" self.layer3 = blockUNet( channels * 2, channels * 2, \"enc_layer3\", transposed=False, bn=True, relu=False, dropout=dropout, )\n",
|
||||
" self.layer4 = blockUNet( channels * 2, channels * 4, \"enc_layer4\", transposed=False, bn=True, relu=False, dropout=dropout, )\n",
|
||||
" self.layer5 = blockUNet( channels * 4, channels * 8, \"enc_layer5\", transposed=False, bn=True, relu=False, dropout=dropout, )\n",
|
||||
" self.layer6 = blockUNet( channels * 8, channels * 8, \"enc_layer6\", transposed=False, bn=True, relu=False, dropout=dropout, size=2, pad=0, )\n",
|
||||
" self.layer7 = blockUNet( channels * 8, channels * 8, \"enc_layer7\", transposed=False, bn=True, relu=False, dropout=dropout, size=2, pad=0, )\n",
|
||||
"\n",
|
||||
" self.layer1 = blockUNet(3 , channels*1, 'enc_layer1', transposed=False, bn=True, relu=False, dropout=dropout )\n",
|
||||
" self.layer2 = blockUNet(channels , channels*2, 'enc_layer2', transposed=False, bn=True, relu=False, dropout=dropout )\n",
|
||||
" self.layer3 = blockUNet(channels*2, channels*2, 'enc_layer3', transposed=False, bn=True, relu=False, dropout=dropout )\n",
|
||||
" self.layer4 = blockUNet(channels*2, channels*4, 'enc_layer4', transposed=False, bn=True, relu=False, dropout=dropout )\n",
|
||||
" self.layer5 = blockUNet(channels*4, channels*8, 'enc_layer5', transposed=False, bn=True, relu=False, dropout=dropout ) \n",
|
||||
" self.layer6 = blockUNet(channels*8, channels*8, 'enc_layer6', transposed=False, bn=True, relu=False, dropout=dropout , size=2,pad=0)\n",
|
||||
" self.layer7 = blockUNet(channels*8, channels*8, 'enc_layer7', transposed=False, bn=True, relu=False, dropout=dropout , size=2,pad=0)\n",
|
||||
" \n",
|
||||
" # note, kernel size is internally reduced by one for the decoder part\n",
|
||||
" self.dlayer7 = blockUNet(channels*8, channels*8, 'dec_layer7', transposed=True, bn=True, relu=True, dropout=dropout , size=2,pad=0)\n",
|
||||
" self.dlayer6 = blockUNet(channels*16,channels*8, 'dec_layer6', transposed=True, bn=True, relu=True, dropout=dropout , size=2,pad=0)\n",
|
||||
" self.dlayer5 = blockUNet(channels*16,channels*4, 'dec_layer5', transposed=True, bn=True, relu=True, dropout=dropout ) \n",
|
||||
" self.dlayer4 = blockUNet(channels*8, channels*2, 'dec_layer4', transposed=True, bn=True, relu=True, dropout=dropout )\n",
|
||||
" self.dlayer3 = blockUNet(channels*4, channels*2, 'dec_layer3', transposed=True, bn=True, relu=True, dropout=dropout )\n",
|
||||
" self.dlayer2 = blockUNet(channels*4, channels , 'dec_layer2', transposed=True, bn=True, relu=True, dropout=dropout )\n",
|
||||
" self.dlayer1 = blockUNet(channels*2, 3 , 'dec_layer1', transposed=True, bn=False, activation=False, dropout=dropout )\n",
|
||||
" self.dlayer7 = blockUNet( channels * 8, channels * 8, \"dec_layer7\", transposed=True, bn=True, relu=True, dropout=dropout, size=2, pad=0, )\n",
|
||||
" self.dlayer6 = blockUNet( channels * 16, channels * 8, \"dec_layer6\", transposed=True, bn=True, relu=True, dropout=dropout, size=2, pad=0, )\n",
|
||||
" self.dlayer5 = blockUNet( channels * 16, channels * 4, \"dec_layer5\", transposed=True, bn=True, relu=True, dropout=dropout, )\n",
|
||||
" self.dlayer4 = blockUNet( channels * 8, channels * 2, \"dec_layer4\", transposed=True, bn=True, relu=True, dropout=dropout, )\n",
|
||||
" self.dlayer3 = blockUNet( channels * 4, channels * 2, \"dec_layer3\", transposed=True, bn=True, relu=True, dropout=dropout, )\n",
|
||||
" self.dlayer2 = blockUNet( channels * 4, channels, \"dec_layer2\", transposed=True, bn=True, relu=True, dropout=dropout, )\n",
|
||||
" self.dlayer1 = blockUNet( channels * 2, 3, \"dec_layer1\", transposed=True, bn=False, activation=False, dropout=dropout, )\n",
|
||||
"\n",
|
||||
" def forward(self, x):\n",
|
||||
" # note, this Unet stack could be allocated with a loop, of course... \n",
|
||||
" out1 = self.layer1(x)\n",
|
||||
" def forward(self, input):\n",
|
||||
" # note, this Unet stack could be allocated with a loop, of course...\n",
|
||||
" out1 = self.layer1(input)\n",
|
||||
" out2 = self.layer2(out1)\n",
|
||||
" out3 = self.layer3(out2)\n",
|
||||
" out4 = self.layer4(out3)\n",
|
||||
@@ -362,9 +300,9 @@
|
||||
"\n",
|
||||
"def weights_init(m):\n",
|
||||
" classname = m.__class__.__name__\n",
|
||||
" if classname.find('Conv') != -1:\n",
|
||||
" if classname.find(\"Conv\") != -1:\n",
|
||||
" m.weight.data.normal_(0.0, 0.02)\n",
|
||||
" elif classname.find('BatchNorm') != -1:\n",
|
||||
" elif classname.find(\"BatchNorm\") != -1:\n",
|
||||
" m.weight.data.normal_(1.0, 0.02)\n",
|
||||
" m.bias.data.fill_(0)\n"
|
||||
]
|
||||
@@ -377,49 +315,46 @@
|
||||
"source": [
|
||||
"Next, we can initialize an instance of the `DfpNet`.\n",
|
||||
"\n",
|
||||
"Below, the `EXPO` parameter here controls the exponent for the feature maps of our Unet: this directly scales the network size (3 gives a network with ca. 150k parameters). This is relatively small for a generative NN with $3 \\times 128^2 = \\text{ca. }49k$ outputs, but yields fast training times and prevents overfitting given the relatively small data set we're using here. Hence it's a good starting point."
|
||||
"Below, the `EXPO` parameter here controls the exponent for the feature maps of our Unet: this directly scales the network size (an exponent of 4 gives a network with ca. 585k parameters). This is a medium sized network for a generative NN with $3 \\times 128^2 = \\text{ca. }49k$ outputs, and still yields fast training times. Hence it's a good starting point. The `weights_init` function initializes the conv net to a reasonable initial value range, so that we can directly train with a fixed learning rate (otherwise learning rate schedules are highly recommended)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "N1uV8k5VIoqT",
|
||||
"outputId": "f91c9beb-5ca9-4516-ae74-7ba87b2a2b0e"
|
||||
"outputId": "bf19c644-7d34-4879-c538-3ffd802d3c4c"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Trainable params: 147363 -> crucial! always keep in view... \n"
|
||||
"Trainable params: 585027 -> crucial! always keep in view... \n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# channel exponent to control network size\n",
|
||||
"EXPO = 3\n",
|
||||
"EXPO = 4\n",
|
||||
"\n",
|
||||
"torch.set_default_device(\"cuda:0\")\n",
|
||||
"device = torch.get_default_device()\n",
|
||||
"\n",
|
||||
"# setup network\n",
|
||||
"net = DfpNet(channelExponent=EXPO)\n",
|
||||
"#print(net) # to double check the details...\n",
|
||||
"\n",
|
||||
"nn_parameters = filter(lambda p: p.requires_grad, net.parameters())\n",
|
||||
"params = sum([np.prod(p.size()) for p in nn_parameters])\n",
|
||||
"\n",
|
||||
"# crucial parameter to keep in view: how many parameters do we have?\n",
|
||||
"print(\"Trainable params: {} -> crucial! always keep in view... \".format(params)) \n",
|
||||
"\n",
|
||||
"net.apply(weights_init)\n",
|
||||
"\n",
|
||||
"criterionL1 = nn.L1Loss()\n",
|
||||
"optimizerG = optim.Adam(net.parameters(), lr=LR, betas=(0.5, 0.999), weight_decay=0.0)\n",
|
||||
"# crucial parameter to keep in view: how many parameters do we have?\n",
|
||||
"nn_parameters = filter(lambda p: p.requires_grad, net.parameters())\n",
|
||||
"print(\"Trainable params: {} -> crucial! always keep in view... \".format( sum([np.prod(p.size()) for p in nn_parameters]) ))\n",
|
||||
"\n",
|
||||
"targets = torch.autograd.Variable(torch.FloatTensor(BATCH_SIZE, 3, 128, 128))\n",
|
||||
"inputs = torch.autograd.Variable(torch.FloatTensor(BATCH_SIZE, 3, 128, 128))\n"
|
||||
"LR = 0.0002 # learning rate\n",
|
||||
"\n",
|
||||
"loss = nn.L1Loss()\n",
|
||||
"optimizer = optim.Adam(net.parameters(), lr=LR, betas=(0.5, 0.999), weight_decay=0.0)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -428,7 +363,8 @@
|
||||
"id": "UNjBAvfWJMTR"
|
||||
},
|
||||
"source": [
|
||||
"With an exponent of 3, this network has 147555 trainable parameters. As the subtle hint in the print statement indicates, this is a crucial number to always have in view when training NNs. It's easy to change settings, and get a network that has millions of parameters, and as a result probably all kinds of convergence and overfitting problems. The number of parameters definitely has to be matched with the amount of training data, and should also scale with the depth of the network. How these three relate to each other exactly is problem dependent, though."
|
||||
"As the subtle hint in the print statement indicates, the parameter count is a crucial number to have in view when training NNs. It's easy to change settings, and get a network that has millions of parameters, and as a result can cause wasting resources at training time (and potentially training instabilities).\n",
|
||||
"The number of parameters definitely has to be matched with the amount of training data, and should also scale with the depth of the network. How exactly these three relate to each other is problem dependent, though."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -450,74 +386,74 @@
|
||||
"base_uri": "https://localhost:8080/"
|
||||
},
|
||||
"id": "kkOYvwxqKa8n",
|
||||
"outputId": "5daf4281-1151-4fc4-fa3a-74a9ff8435d9"
|
||||
"outputId": "ee34c1ec-7b83-41e8-eafb-15814fa4cd64"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training from scratch\n",
|
||||
"Epoch: 0, L1 train: 0.29219, L1 vali: 0.23295\n",
|
||||
"Epoch: 1, L1 train: 0.25406, L1 vali: 0.22507\n",
|
||||
"Epoch: 2, L1 train: 0.22487, L1 vali: 0.21019\n",
|
||||
"Epoch: 20, L1 train: 0.05228, L1 vali: 0.04134\n",
|
||||
"Epoch: 40, L1 train: 0.03730, L1 vali: 0.03020\n",
|
||||
"Epoch: 60, L1 train: 0.03236, L1 vali: 0.02523\n",
|
||||
"Epoch: 80, L1 train: 0.03364, L1 vali: 0.02302\n",
|
||||
"Training done, saved network\n"
|
||||
"Training from scratch...\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"loss train: 0.02743, loss val: 0.03572: 7%|█▎ | 14/200 [00:23<05:01, 1.62s/it]"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"history_L1 = []\n",
|
||||
"history_L1val = []\n",
|
||||
"EPOCHS = 200 # number of training epochs\n",
|
||||
"\n",
|
||||
"if os.path.isfile(\"network\"):\n",
|
||||
" print(\"Found existing network, loading & skipping training\")\n",
|
||||
" net.load_state_dict(torch.load(\"network\")) # optionally, load existing network\n",
|
||||
"loss_hist = []\n",
|
||||
"loss_hist_val = []\n",
|
||||
"\n",
|
||||
"if os.path.isfile(\"dfpnet\"): # NT_DEBUG\n",
|
||||
" print(\"Found existing network, loading & skipping training\")\n",
|
||||
" net.load_state_dict(torch.load(\"dfpnet\"))\n",
|
||||
"\n",
|
||||
"else:\n",
|
||||
" print(\"Training from scratch\")\n",
|
||||
" for epoch in range(EPOCHS):\n",
|
||||
" net.train()\n",
|
||||
" L1_accum = 0.0\n",
|
||||
" for i, traindata in enumerate(trainLoader, 0):\n",
|
||||
" inputs_curr, targets_curr = traindata\n",
|
||||
" inputs.data.copy_(inputs_curr.float())\n",
|
||||
" targets.data.copy_(targets_curr.float())\n",
|
||||
" print(\"Training from scratch...\")\n",
|
||||
" pbar = tqdm(initial=0, total=EPOCHS, ncols=96)\n",
|
||||
" for epoch in range(EPOCHS):\n",
|
||||
"\n",
|
||||
" net.zero_grad()\n",
|
||||
" gen_out = net(inputs)\n",
|
||||
" # training\n",
|
||||
" net.train()\n",
|
||||
" loss_acc = 0\n",
|
||||
" for i, (inputs, targets) in enumerate(loader_train):\n",
|
||||
" inputs = inputs.float()\n",
|
||||
" targets = targets.float() \n",
|
||||
"\n",
|
||||
" lossL1 = criterionL1(gen_out, targets)\n",
|
||||
" lossL1.backward()\n",
|
||||
" optimizerG.step()\n",
|
||||
" L1_accum += lossL1.item()\n",
|
||||
" net.zero_grad()\n",
|
||||
" outputs = net(inputs)\n",
|
||||
" lossL1 = loss(outputs, targets)\n",
|
||||
" lossL1.backward()\n",
|
||||
" optimizer.step()\n",
|
||||
" loss_acc += lossL1.item()\n",
|
||||
"\n",
|
||||
" # validation\n",
|
||||
" net.eval()\n",
|
||||
" L1val_accum = 0.0\n",
|
||||
" for i, validata in enumerate(valiLoader, 0):\n",
|
||||
" inputs_curr, targets_curr = validata\n",
|
||||
" inputs.data.copy_(inputs_curr.float())\n",
|
||||
" targets.data.copy_(targets_curr.float())\n",
|
||||
" loss_hist.append(loss_acc / len(loader_train))\n",
|
||||
"\n",
|
||||
" outputs = net(inputs)\n",
|
||||
" outputs_curr = outputs.data.cpu().numpy()\n",
|
||||
" # evaluate validation samples\n",
|
||||
" net.eval()\n",
|
||||
" loss_acc_v = 0\n",
|
||||
" with torch.no_grad():\n",
|
||||
" for i, (inputs, targets) in enumerate(loader_val):\n",
|
||||
" inputs = inputs.float()\n",
|
||||
" targets = targets.float()\n",
|
||||
"\n",
|
||||
" lossL1val = criterionL1(outputs, targets)\n",
|
||||
" L1val_accum += lossL1val.item()\n",
|
||||
" outputs = net(inputs)\n",
|
||||
" loss_acc_v += loss(outputs, targets).item()\n",
|
||||
"\n",
|
||||
" # data for graph plotting\n",
|
||||
" history_L1.append( L1_accum / len(trainLoader) )\n",
|
||||
" history_L1val.append( L1val_accum / len(valiLoader) )\n",
|
||||
" loss_hist_val.append(loss_acc_v / len(loader_val))\n",
|
||||
" pbar.set_description(\"loss train: {:7.5f}, loss val: {:7.5f}\".format( loss_hist[-1], loss_hist_val[-1] ) , refresh=False); pbar.update(1)\n",
|
||||
"\n",
|
||||
" if epoch<3 or epoch%20==0:\n",
|
||||
" print( \"Epoch: {}, L1 train: {:7.5f}, L1 vali: {:7.5f}\".format(epoch, history_L1[-1], history_L1val[-1]) )\n",
|
||||
" torch.save(net.state_dict(), \"dfpnet\")\n",
|
||||
" print(\"training done, saved network weights\")\n",
|
||||
"\n",
|
||||
" torch.save(net.state_dict(), \"network\" )\n",
|
||||
" print(\"Training done, saved network\")\n"
|
||||
"loss_hist = np.asarray(loss_hist)\n",
|
||||
"loss_hist_val = np.asarray(loss_hist_val)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -526,9 +462,9 @@
|
||||
"id": "4KuUpJsSL3Jv"
|
||||
},
|
||||
"source": [
|
||||
"The NN is finally trained! The losses should have nicely gone down in terms of absolute values: With the standard settings from an initial value of around 0.2 for the validation loss, to ca. 0.02 after 100 epochs. \n",
|
||||
"The NN is finally trained! The losses should have nicely gone down in terms of absolute values: With the standard settings from an initial value of around 0.2 for the validation loss, to ca. 0.01 after all epochs.\n",
|
||||
"\n",
|
||||
"Let's look at the graphs to get some intuition for how the training progressed over time. This is typically important to identify longer-term trends in the training. In practice it's tricky to spot whether the overall trend of 100 or so noisy numbers in a command line log is going slightly up or down - this is much easier to spot in a visualization."
|
||||
"Let's look at the graphs to get some intuition for how the training progressed over time. This is typically important to identify longer-term trends in the training. In practice it's tricky to spot whether the overall trend of hundreds of noisy numbers in a command line log is going slightly up or down - this is much easier to spot in a visualization."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -537,35 +473,17 @@
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 286
|
||||
"height": 450
|
||||
},
|
||||
"id": "Hlki3bU8JMTT",
|
||||
"outputId": "fa4b4911-85f2-478d-87e2-44b90bf5b83b"
|
||||
"outputId": "3a9fa5a9-6682-4857-d9e4-656b252bd78b"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1d3H8c8vk8lGyB7WsARJWBMSCKAiCOICaAEVFOpGURGXWrGPllarFuvTPnWpVdGKu1aLu4KCiriLCmGRfQkQJBAgBBKyLzPn+eMMIewhJEyY+b1fr7zI3GXmdzP6vfeee+65YoxBKaWU7wrwdgFKKaUalwa9Ukr5OA16pZTycRr0Sinl4zTolVLKxwV6u4BDxcXFmY4dO3q7DKWUOq0sXrx4tzEm/kjz6hT0IjIM+BfgAJ43xvz9kPmTgVsBF1AMTDLGrPbM+yNwvWfe7caYT4/1WR07diQzM7MuZSmllPIQkS1Hm3fcphsRcQDTgeFAd2C8iHQ/ZLE3jDEpxpg04B/AY551uwPjgB7AMOBpz/sppZQ6RerSRt8PyDLGbDLGVAIzgVG1FzDG7Kv1shmw/y6sUcBMY0yFMWYzkOV5P6WUUqdIXZpu2gJba73OAfofupCI3ArcCQQB59Va98dD1m1br0qVUkrVS4NdjDXGTAemi8ivgXuB6+q6rohMAiYBtG/fvqFKUkrVUVVVFTk5OZSXl3u7FHUcISEhJCQk4HQ667xOXYJ+G9Cu1usEz7SjmQk8cyLrGmNmADMAMjIydPAdpU6xnJwcmjdvTseOHRERb5ejjsIYQ35+Pjk5OSQmJtZ5vbq00S8CkkQkUUSCsBdXZ9VeQESSar28GNjg+X0WME5EgkUkEUgCFta5OqXUKVFeXk5sbKyGfBMnIsTGxp7wmddxj+iNMdUichvwKbZ75YvGmFUiMg3INMbMAm4TkfOBKmAvnmYbz3JvAauBauBWY4zrhCpUSp0SGvKnh/p8T3VqozfGzAHmHDLtvlq//+4Y6z4EPHTClZ2gggL4179gxAjo27exP00ppU4fPjUEwgMPwDffeLsKpdSJys/PJy0tjbS0NFq1akXbtm1rXldWVh5z3czMTG6//fbjfsbZZ5/dILV+9dVXXHLJJQ3yXqdKkxsCob4iIyE8HLZuPf6ySqmmJTY2lmXLlgHwwAMPEB4ezv/8z//UzK+uriYw8MhxlZGRQUZGxnE/Y8GCBQ1T7GnIZ47oRSAhAXJyvF2JUqohTJgwgcmTJ9O/f3/uvvtuFi5cyFlnnUV6ejpnn30269atAw4+wn7ggQeYOHEigwcPplOnTjzxxBM17xceHl6z/ODBgxkzZgxdu3blqquuYv+T9ubMmUPXrl3p06cPt99++3GP3Pfs2cPo0aNJTU3lzDPPZPny5QB8/fXXNWck6enpFBUVkZuby6BBg0hLS6Nnz558++23Df43OxqfOaIHDXqlGsIdd4Dn4LrBpKXB44+f+Ho5OTksWLAAh8PBvn37+PbbbwkMDOTzzz/nT3/6E+++++5h66xdu5Yvv/ySoqIiunTpws0333xYn/OlS5eyatUq2rRpw4ABA/j+++/JyMjgpptu4ptvviExMZHx48cft77777+f9PR0PvjgA7744guuvfZali1bxiOPPML06dMZMGAAxcXFhISEMGPGDC666CLuueceXC4XpaWlJ/4HqSefC/p587xdhVKqoYwdOxaHww6PVVhYyHXXXceGDRsQEaqqqo64zsUXX0xwcDDBwcG0aNGCnTt3kpCQcNAy/fr1q5mWlpZGdnY24eHhdOrUqaZ/+vjx45kxY8Yx6/vuu+9qdjbnnXce+fn57Nu3jwEDBnDnnXdy1VVXcdlll5GQkEDfvn2ZOHEiVVVVjB49mrS0tJP625wInwr6du0gNxeqq+EozXlKqeOoz5F3Y2nWrFnN73/+858ZMmQI77//PtnZ2QwePPiI6wQHB9f87nA4qK6urtcyJ2Pq1KlcfPHFzJkzhwEDBvDpp58yaNAgvvnmGz7++GMmTJjAnXfeybXXXtugn3s0PtNGD/aI3u22Ya+U8i2FhYW0bWuHynr55Zcb/P27dOnCpk2byM7OBuDNN9887joDBw7k9ddfB2zbf1xcHBEREWzcuJGUlBT+8Ic/0LdvX9auXcuWLVto2bIlN954IzfccANLlixp8G04Gp8LetB2eqV80d13380f//hH0tPTG/wIHCA0NJSnn36aYcOG0adPH5o3b05kZOQx13nggQdYvHgxqampTJ06lVdeeQWAxx9/nJ49e5KamorT6WT48OF89dVX9OrVi/T0dN58801+97uj3n7U4GT/1eamIiMjw9T3wSMrVkBqKrz5JlxxRQMXppQPW7NmDd26dfN2GV5XXFxMeHg4xhhuvfVWkpKSmDJlirfLOsyRvi8RWWyMOWI/Uz2iV0opj+eee460tDR69OhBYWEhN910k7dLahA+dckyKgrCwjTolVL1M2XKlCZ5BH+yfOqIXsT2vNGgV0qpA3wq6ME23+gwCEopdYBPBr0e0Sul1AE+GfT7b5pSSinlg0Hfrh24XLBjh7crUUrV1ZAhQ/j0008Pmvb4449z8803H3WdwYMHs78r9ogRIygoKDhsmQceeIBHHnnkmJ/9wQcfsHr16prX9913H59//vmJlH9ETWk4Y58Leu1iqdTpZ/z48cycOfOgaTNnzqzTwGJgR52Mioqq12cfGvTTpk3j/PPPr9d7NVUa9EoprxszZgwff/xxzUNGsrOz2b59OwMHDuTmm28mIyODHj16cP/99x9x/Y4dO7J7924AHnroIZKTkznnnHNqhjIG20e+b9++9OrVi8svv5zS0lIWLFjArFmzuOuuu0hLS2Pjxo1MmDCBd955B4D58+eTnp5OSkoKEydOpKKioubz7r//fnr37k1KSgpr16495vZ5ezhjn+pHD7bpBjTolaqvOz65g2U7Gnac4rRWaTw+7OijpcXExNCvXz/mzp3LqFGjmDlzJldccQUiwkMPPURMTAwul4uhQ4eyfPlyUlNTj/g+ixcvZubMmSxbtozq6mp69+5Nnz59ALjsssu48cYbAbj33nt54YUX+O1vf8vIkSO55JJLGDNmzEHvVV5ezoQJE5g/fz7Jyclce+21PPPMM9xxxx0AxMXFsWTJEp5++mkeeeQRnn/++aNun7eHM/a5I/roaAgN1S6WSp1uajff1G62eeutt+jduzfp6emsWrXqoGaWQ3377bdceumlhIWFERERwciRI2vmrVy5koEDB5KSksLrr7/OqlWrjlnPunXrSExMJDk5GYDrrruOb2o9q/Syyy4DoE+fPjUDoR3Nd999xzXXXAMceTjjJ554goKCAgIDA+nbty8vvfQSDzzwACtWrKB58+bHfO+68Lkjen3SlFIn51hH3o1p1KhRTJkyhSVLllBaWkqfPn3YvHkzjzzyCIsWLSI6OpoJEyZQXl5er/efMGECH3zwAb169eLll1/mq6++Oql69w91fDLDHJ+q4Yx97ogeNOiVOh2Fh4czZMgQJk6cWHM0v2/fPpo1a0ZkZCQ7d+5k7ty5x3yPQYMG8cEHH1BWVkZRURGzZ8+umVdUVETr1q2pqqqqGVoYoHnz5hQVFR32Xl26dCE7O5usrCwAXnvtNc4999x6bZu3hzP2uSN6sO30J7mzVkp5wfjx47n00ktrmnD2D+vbtWtX2rVrx4ABA465fu/evbnyyivp1asXLVq0oG/fvjXzHnzwQfr37098fDz9+/evCfdx48Zx44038sQTT9RchAUICQnhpZdeYuzYsVRXV9O3b18mT55cr+3a/yzb1NRUwsLCDhrO+MsvvyQgIIAePXowfPhwZs6cycMPP4zT6SQ8PJxXX321Xp9Zm08NU7zfPffA//0fVFSA5ylkSqlj0GGKTy9+PUzxfgkJ9qapnTu9XYlSSnmfTwa9drFUSqkDfDLo9980pV0slaq7ptaMq46sPt+TBr1SipCQEPLz8zXsmzhjDPn5+YSEhJzQenXqdSMiw4B/AQ7geWPM3w+ZfydwA1AN5AETjTFbPPNcwArPor8YY0bSyGJjoVkzOM49DEopj4SEBHJycsjLy/N2Keo4QkJCSNh/NFtHxw16EXEA04ELgBxgkYjMMsbUvj1tKZBhjCkVkZuBfwBXeuaVGWPSTqiqkyQCnTvDhg2n8lOVOn05nU4SExO9XYZqJHVpuukHZBljNhljKoGZwKjaCxhjvjTG7B+Q4UfgxHY3jSApSYNeKaWgbkHfFqjd2p3jmXY01wO1b18LEZFMEflRREYfaQURmeRZJrOhTh2TkmDzZn0AiVJKNejFWBG5GsgAHq41uYOnE/+vgcdF5IxD1zPGzDDGZBhjMuLj4xuklqQkG/LaTq+U8nd1CfptQLtarxM80w4iIucD9wAjjTEV+6cbY7Z5/t0EfAWkn0S9dZaUZP/1DFOhlFJ+qy5BvwhIEpFEEQkCxgGzai8gIunAs9iQ31VrerSIBHt+jwMGAEcfY7QB7Q96badXSvm74/a6McZUi8htwKfY7pUvGmNWicg0INMYMwvbVBMOvC0icKAbZTfgWRFxY3cqfz+kt06jadECwsM16JVSqk796I0xc4A5h0y7r9bvR3zAojFmAZByMgXWl4j2vFFKKfDRO2P306BXSik/CPrsbKiq8nYlSinlPT4f9C6X7U+vlFL+yueDHrT5Rinl3zTolVLKx/lU0H+x+QsKywtrXsfFQWSkBr1Syr/5TNCvz1/P0FeH8vCCA6MvaBdLpZTyoaBPjk1mXM9x/PPHf5JblFszPSlJh0FQSvk3nwl6gAeHPEilq5IHv3mwZlpSEmzZApWVXixMKaW8yKeCvnNMZ27sfSPPLXmOrD32MD4pCdxu2LTJy8UppZSX+FTQA9x37n0EOYK494t7AfukKdB2eqWU//K5oG8V3ooLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 432x288 with 1 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"needs_background": "light"
|
||||
},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"l1train = np.asarray(history_L1)\n",
|
||||
"l1vali = np.asarray(history_L1val)\n",
|
||||
"\n",
|
||||
"plt.plot(np.arange(l1train.shape[0]),l1train,'b',label='Training loss')\n",
|
||||
"plt.plot(np.arange(l1vali.shape[0] ),l1vali ,'g',label='Validation loss')\n",
|
||||
"plt.plot(np.arange(loss_hist.shape[0]), loss_hist, \"b\", label=\"Training loss\")\n",
|
||||
"plt.plot(np.arange(loss_hist_val.shape[0]), loss_hist_val, \"g\", label=\"Validation loss\")\n",
|
||||
"plt.legend()\n",
|
||||
"plt.show()\n"
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -579,12 +497,12 @@
|
||||
"## Training progress and validation\n",
|
||||
"\n",
|
||||
"If you look closely at this graph, you should spot something peculiar:\n",
|
||||
"_Why is the validation loss lower than the training loss_?\n",
|
||||
"_Why is the validation loss lower than the training loss in the initial phase_?\n",
|
||||
"The data is similar to the training data of course, but in a way it's slightly \"tougher\", because the network certainly never received any validation samples during training. It is natural that the validation loss slightly deviates from the training loss, but how can the L1 loss be _lower_ for these inputs?\n",
|
||||
"\n",
|
||||
"This is caused by the way the the training loop above is implemented in pytorch: while the training loss is evaluated in training mode via `net.train()`, the evaluation takes place after a call to `net.eval()`. This turns off batch normalization, and would disable features like dropout (if active). This slightly changes the evaluation. The code also runs a training step, and the loss for each point in the graph is measured with the evolving state of the network in an epoch. The network is updated, and afterwards runs through the validation samples. Thus all validation samples are using a state that is slightly different (and hopefully a bit better) than the initial states of the epoch. Due to both reasons, the validation loss can deviate, and in this example it's typically slightly lower.\n",
|
||||
"This is caused by the way the the training loop above is implemented in pytorch: the training loss for each point in the graph is measured with the evolving state of the network in an epoch. The network is updated, and afterwards runs through the validation samples. Thus all validation samples are using a state that is slightly different (and hopefully a bit better) than the initial states of the epoch. Due to both reasons, the validation loss can deviate, and in this example it's slightly lower initially, while later on the network starts to slow down in terms of improving the new conditions of the validation samples. This indicates a slight overfitting for the training data, but nonetheless the validation loss still increases.\n",
|
||||
"\n",
|
||||
"A general word of caution here: never evaluate your network with training data! That won't tell you much because overfitting is a very common problem. At least use data the network hasn't seen before, i.e. validation data, and if that looks good, try some more different (at least slightly out-of-distribution) inputs, i.e., _test data_. The next cell runs the trained network over the validation data, and displays one of them with the `showSbs` function.\n",
|
||||
"A general word of caution here: never evaluate your network with training data! That won't tell you much because overfitting is a very common problem. At least use data the network hasn't seen before, i.e. validation data, and if that looks good, try some more different (at least slightly out-of-distribution) inputs, i.e., _test data_. The next cell runs the trained network on a batch of samples from the validation data, and displays one with the `plot` function.\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
@@ -594,36 +512,25 @@
|
||||
"metadata": {
|
||||
"colab": {
|
||||
"base_uri": "https://localhost:8080/",
|
||||
"height": 353
|
||||
"height": 483
|
||||
},
|
||||
"id": "N6OONK0bL4ev",
|
||||
"outputId": "99dcb709-c2c3-4407-f0d4-9e9a91cee3bb",
|
||||
"outputId": "1a473bd9-b3de-463a-cff9-3d886afeda93",
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAE8CAYAAABQAcL1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9ebQsy1Xe+e3Iqjrn3vsmvafBQgNC4omZxjYWk3HTDRgWxg1msZgbZNO4GUwbDLhxNxhauHvRg+02y0yCZQFuMa5ls2SgQdBgy8igCZtBMggJEO8J6U33vfvucM6pyozdf0RE5o7IiMjIrKozvdx31T1VmZGRkVlZuX/57R0RxMyYbbbZZpttttlmU2fdgNlmm2222Wab7XzYDAWzzTbbbLPNNhuAGQpmm2222WabbTZrMxTMNttss80222wAZiiYbbbZZpttttmszVAw22yzzTbbbLMBmKFgtktqRMRE9MH2/Q8Q0beXlJ2wny8lotdPbed5t23OzWyzzXbxbIaC2c6lEdEvEtGrIss/h4jeT0SL0rqY+auZ+bt20KaXWCfZ7puZX8vMf3Xbui+jEdGXENF7iOg2Ef0sEd1/1m2abbbZ8jZDwWzn1X4UwJcREQXL/1sAr2Xm+gzaNFuhEdFHAPhBmO/reQDuAPi+M23UbLPNNmgzFMx2Xu1nATwA4JPdAiJ6FoDPBvBjRPQKIvoNInqKiN5HRP+ciFaxiojoR4joH4nP32K3+TMi+ltB2b9GRP+RiJ4mooeI6DvF6jfYv08R0S0i+gQieiUR/brY/hOJ6C1EdMP+/USx7t8S0XcR0RuJ6CYRvZ6Inp1o87OJ6Ofs8V0non9PRMqu+1Yieret4x1E9DfEdq+09f9Tu+0f2Ta90h7Po0T0FcG5+QEi+mVb378jog9MtOmAiP4vIvpTInrEbnclVhbAlwL4N8z8Bma+BeDbAXweEd2dKD/bbLOdA5uhYLZzacx8BOCnAXy5WPwFAH6fmX8bQAPgGwE8G8AnAPhUAF87VC8RfSaAbwbw6QAeBPBpQZHbdp/3AfhrAL6GiD7Xrvsr9u99zHwXM/9GUPf9AH4ewPfAAM0/AfDzRPSAKPYlAP4mgOcCWNm2xOybADwM4DkwT9r/EwA3Jvm7YWDpXgD/C4D/h4ieL7b9OAC/Y9vw4wB+EsBfAvDBAL4MwD8nortE+S8F8F0w5/I/AXhtok3fDeDlAD7G1vUCAP8wUfYjAPy2+8DM7wawttvPNtts59RmKJjtPNuPAvh8Ijq0n7/cLgMzv42Zf5OZa2b+Exip+r8sqPMLALyGmX+PmW8D+E65kpn/LTP/LjNrZv4dAD9RWC9gIOIPmflf2nb9BIDfB/DXRZnXMPM7BfR8TKKuDYDnA/hAZt4w879nO1EJM/8MM/+ZbeNPAfhDAK8Q2/4xM7+GmRsAPwXgRQBexcwnzPx6GOcskwd/3j7RnwD4nwF8AhG9SDbGhnH+NoBvZObrzHwTwP8G4IsS7b8LwI1g2Q0As1Iw22zn2GYomO3cGjP/OoDHAXwuEb0MxvH9OAAQ0cutvP5+InoaxkFFpfjAPgDAQ+Lze+RKIvo4Ivo1InqMiG4A+OrCel3d7wmWvQfmidrZ+8X7OzDOM2b/J4B3AXi9DQF8q2jjlxPRf7LhgacAfGTQxkfE+yMAYOZwmdxvez6s1H/dHou05wC4CuBtYr+/aJfH7BaAe4Jl9wC4mSg/22yznQOboWC2824/BqMQfBmAXxLO7fthnsIfZOZ7YOT1MCkxZu+DeXJ29uJg/Y8DeB2AFzHzvQB+QNQ7NKXonwEI4/EvBvDegnZ5xsw3mfmbmPmlAP4bAH+PiD7Vxvt/CMDfAfAAM98H4PdQduwpa8+HDSvcD3Ms0h6HgYmPYOb77OteZk5BzdsB/Bei3pcCOADwzi3aOdtss+3ZZiiY7bzbj8HE/b8KNnRg7W4ATwO4RUQfCuBrCuv7aQCvJKIPJ6KrAL4jWH83gOvMfExEr4DJAXD2GAAN4KWJun8BwMttV7wFEX0hgA8H8HOFbWuNiD6biD7YyvY3YHIoNIBrMHDymC33N2GUgm3ss4joL9tEze8C8JvMLNUUMLOGgZF/SkTPtft+ARF9RqLO1wL460T0yUR0DcCrAPwrG3aYbbbZzqnNUDDbuTabL/AfYJzh68Sqb4Zx2DdhnNVPFdb3/wL4vwH8Kow8/6tBka8F8CoiugmTRPfTYts7AP5XAG+0EvrHB3U/AdM74psAPAHg7wP4bGZ+vKRtgT0I4FdgZPjfAPB9zPxrzPwOAP/YLnsEwEcBeOOE+qX9OAwcXQfwF2FUmZj9jzDn7DdtyOZXAHxIrCAzvx0m9PJaAI/CwNZgIuhss812tkY2d2m22WZ7BhoR/QiAh5n52866LbPNNtvZ26wUzDbbbLPNNttsAGYomG222WabbbYLZ0T0L+xgZL+XWE9E9D1E9C4i+h0i+gtF9c7hg9lmm2222Wa7WEZEfwUm5+jHmLmXbExEnwXg6wF8FsyAZv+MmT9uqN5ZKZhtttlmm222C2bM/AaY5OCUfQ4MMDAz/yaA+4KRT6M2Q8Fss80222yzXT57AfyB2h6GP5Ba1Iqnnx1jz773Kr/kefeZD6OiEyNDGRc58lE81ExBwW2GrSm13mSFW9id9cC+JtTZ26awkrH7Oo1zHdrY67y4PGc/7mJfU8KTDz0anddq57bN7WPMYZUWTZbjgjKRtXv4Os+FPd28/3FmTo2keS7tMz7jFfzEE+Go33l729ve+XYAx2LRq5n51TttWMT2AgUved59eMv3/HeAHnGJjSqrxzdq6r6GTG3hJVShUDO0j8H1A/spOQYqaGvhudC/9Sdim4I6wmUBoFDJNlPLlKxry2whvJVc06nrtnR5rJxYxuH6mOcbqjM4jF6dBe34hu97Sa7K0TbkwGPNCRfpoJLeqcrUyWKttzxz6iRMyW20qIsTy+N1h+3vH3TsNIXb+fuYfh/N1StNCQrXYCgQfunG/x4OJX7u7YknbuBNb/7BUdssqv/qmJk/dovdvhf+6K0vRMHoqhcvfDAVCDR3r522h6fXrfV2zqB4/ba31QtmQ85eUTkQxMp661X32sZkPam6Um0ZaqMsl7EeXO1SHcrZwPWrMq8SI8ofSuy0hItUUEHvVBW25TRsLBAwxgEBM48CAg3uvaZsm2vTuTdGd78vfW1vrwPw5bYXwscDuMHM7xvaaC9KwWgrdaZjT9SuAWDsPktVBK2HnYrm7VSJbWyHKoG/TUmZkSrB2M9jlrXrToGl3T5i17xrW3h9K+pff1PK7NN2vC/5TQzdHdylE/NnsdNA8J2lIvIc6ranjmicilGqEnjbFwDB0DbdPocP9sI67b0b7/zhjIh+AsCnAHg2ET0MMyrpEgCY+Qdghl3/LJhRSO/ATNk+aPuDgmJHvwcgOAsYiJlrR4nDdMeXczg5MDhLaDgvtm8gKIGBKU/XuZut3Gf4G9iD0ydFccn/tGyL+2b47aSqSjnjKWCQKivrIpAXQhiybWT53Kb7BIIZBgpsx1DAzF88sJ4BfN3Yes9WKdg1EJwXGAhtLBxMfRLNQsMW9Z5Ti+YSpGybnILceduFxB7WkboBKxUHAyAPAmM/h23j6ZAxxUqeoEvMfWuxu0dKNRh7ePsuv41l8wEKy+dgYAaBEcbYzUV9CrYfKNjlsZ8WEPAIiiuR02NWCgc5B36ZFYEhJz0mMDw2OTMKDYnvYN+xdll/z2slwgo7BIFTVQtSbSg4xaX32LFwEJ6aqWrBviwVOihVCcJiY9WBsTkBuzB1rrI2ptjuwwf7srNTCkpuOrtIwovZGAAo3X4MKJQ49qlgMAUaLiBoZFWCbfMMpsLANucw96QOxOFgl2Aw9RH2tHMShMW+jmwkxv5NwcFUMEidgl2emrHOVZZPAUyq3m1gYF/qwaVQJWYo2NJ2DQTbgsCY+ksAYVswOE9W6gz3dSw7BYRIG1MwsEuQCuvqBbdjj7QR1WAfiYQT9fye2nAK8JATWZyl4GAsGIxq18i8grFWmng41IIYEAw55EvhsPdtjBkKsrZtF7rzBAO5fQ7BQUk4IQUGkxSBCwIZMSuV7LcChODcbAEDQ/kOg/J8qodBzEGHqkEOBAoh4VRCCIkxDnLy/BgbAoQYHIwBg5RasKsQQnT8hELn39VRHjYYAwQzCIy1OXww3XYFBGcBA7k25ABhqnyf2u4ChgPGWHGC4ShAEN9PDAYydY1KeMxsE3XCMThIqQYpMCgNG5xhKCC0cFyAlJXCQ7Y7IsaBwRg7jVOaAoVtgGBbGNiFKkIXPo9A2KwUZGybX0jJtlNhYGo8tdRYTweDi/yE72xs+6c4/lII2BEQTIGBnMn6eoCQgoOpYHDaNnIkxFJLwUMyETCVooFyMChRC/JtHnf4MUc85kk9BwSl6kA+JLGf6ypW78UFBQadhwfVAjtfSkGOpHYJBDvL/kncuFM2FFbIhRPGhhFOSy04r4pECgLGAEHk2AZBYAq8Ra57t58oHORUg1gCYmzbXQGDP7rOuTEJCzFAiIotrrwoMxYM2vrRd8b7tpLkwhwQjIGBfeZH5Oys9rsTm5WCiOVuPNsAwWnDwFD9u4CDMWBw3u00EgzHvg/bRZlyyMDALo4tM0BRVD0YUg0kGJxRR/psPkLmMXkfeQwOEFJwkFMNxuZYZod82DLZMNaO+PQU8bDBGCDYBQxMmd/g0hrj3ITmhux8KQVTrAQITvvLCOO3KcuFFMaAwRhV4CKChXDYO5PsC4EgPtnSDiaYctZTAvKAwOG1NRYMRigEO0s2HFnHtt9wam8p9SClGsTAYIxa0O13+BSE4YuclT7RlwBBeV35A9gm8TC17eWCBZ6Vgp7tQyUYAoIpN7QhuXbKGAC57aaAQWnZ85BweNoAUqQYTASC3ERFUy3XFTECCL2wQqgajBnDd8z6XdmOcglLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1152x360 with 2 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"needs_background": "light"
|
||||
},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"net.eval()\n",
|
||||
"for i, validata in enumerate(valiLoader, 0):\n",
|
||||
" inputs_curr, targets_curr = validata\n",
|
||||
" inputs.data.copy_(inputs_curr.float())\n",
|
||||
" targets.data.copy_(targets_curr.float())\n",
|
||||
" \n",
|
||||
" outputs = net(inputs)\n",
|
||||
" outputs_curr = outputs.data.cpu().numpy()\n",
|
||||
" if i<1: showSbs(targets_curr[0] , outputs_curr[0], title=\"Validation sample %d\"%(i*BATCH_SIZE))\n"
|
||||
"inputs, targets = next(iter(loader_val))\n",
|
||||
"inputs = inputs.float()\n",
|
||||
"targets = targets.float()\n",
|
||||
"\n",
|
||||
"outputs = net(inputs)\n",
|
||||
"\n",
|
||||
"outputs = outputs.data.cpu().numpy()\n",
|
||||
"inputs = inputs.cpu()\n",
|
||||
"targets = targets.cpu()\n",
|
||||
"plot(targets[0], outputs[0], mask=inputs[0][2], title=\"Validation sample\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -632,7 +539,7 @@
|
||||
"id": "TJtxXgGoJMTV"
|
||||
},
|
||||
"source": [
|
||||
"Visually, there should at least be a rough resemblance here between input out network output. We'll save the more detailed evaluation for the test data, though."
|
||||
"This shows a good resemblance here between input out network output. The region around the airfoil is typically still a bit noisy (this is caused by the Dirichlet boundary, and could be alleviated with a modified loss and larger networks). The pressure values are typically the most difficult ones to learn. We'll save the more detailed evaluation for the test data, though."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -659,85 +566,30 @@
|
||||
"height": 1000
|
||||
},
|
||||
"id": "dD5O58L9HLE-",
|
||||
"outputId": "89c56574-3010-4069-9664-a59894f5dec1"
|
||||
"outputId": "25a66fab-38e3-438f-c2b7-71fbcbf2e3f6"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Downloading test data, this should be fast...\n",
|
||||
"Loaded 10/10 test samples\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Average test error: 0.028802116494625808\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAE8CAYAAABQAcL1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9e7BtX1bf9R1zrb3Puff+nk033dBp6KYhYMSiSlKBhNIkJhqCRpCkigDREJ8pgSRqjJSRgFr8oWWipEKBxAqUCcpTIgIikbwslaQBK0gTiDwC/eLXv/49+nd/995z9l5rDv8Yc8w11lxzvfbjnL3PXePWuXvvtdea67HXmvMzv2PMMYmZsdhiiy222GKLLeZu+wAWW2yxxRZbbLHTsAUKFltsscUWW2wxAAsULLbYYosttthiwRYoWGyxxRZbbLHFACxQsNhiiy222GKLBVugYLHFFltsscUWA7BAwWKL3Tkjon9MRL/3to9jscUWOz9boGCxO2tE9Kb580T0xHz+yh3K+9tE9G8e41hPxYjo9xDRLxDRYyL6W0T0qbd9TIstttjN2QIFi91ZY+Zn9A/ArwP4A2bZd9328Z2aEdFbAfxPAL4ewFsA/BSA77nVg1psscVu1BYoWOypMyJyRPR1RPTLRPQKEX0vEb0lfHdJRH8tLH+diN5HRG8nom8C8M8A+EtBafhLmXKz24bv/hgR/UMiekhEv0JE/47Z7ncR0QeJ6M8Q0UeJ6CNE9CVE9EVE9I+I6FUi+o/N+t9IRN9PRN8TyvsZIvqcueeasS8F8H5m/j5mvgLwjQA+h4g+a8dLvdhii52ZLVCw2NNoXwvgSwD8TgCfDOA1AN8SvvujAJ4H8C4AnwDgjwN4wsx/FsD/AeBrgtLwNZlys9uG7z4K4F8C8ByAPwbgvyaif9ps+w4AlwDeCeDPAfjLAP4IgM+FwMjXE9F7zPpfDOD7ID36/wHAXyei1cxzTe2fBPAP9AMzPwLwy2H5Yost9hTYAgWLPY32xwH8WWb+IDNfQ3rEf4iISgBbSIP+6cxcM/NPM/MbE8vt3ZaZf4SZf5nF/g6AH4c09nbbb2LmLYDvBvBWAN/MzA+Z+f0Afh6AVQN+mpm/P6z/FyBA8fkzzzW1ZwB8PFn2cQDPTjz/xRZb7MwtVzEstthdt08F8INE5M2yGsDbAfxVSE//u4noBQB/DdKobieU27stEf1+AN8A4DdDYPw+gP/XbPsKM9fhvaoLL5nvn0AabbUP6Btm9kT0QYgSMOdcP5Ss+yZEybD2HICHmXIXW2yxO2iLUrDY02gfAPD7mfkF83fJzB9i5i0z/6fM/FsA/A6I5P+vhe0GpxTt25aILgD8AID/CsDbmfkFAD8KgPY4h3fpGyJyAH4TgA/POdfMuu+HUSOI6AGA94bliy222FNgCxQs9jTatwH4Jh1uR0RvI6IvDu9/NxH9U0RUAHgDIutrL/slAJ/WV+jAtmsAFwBeBlAF1eBf2PMcPpeIvjS4Af4UgGsAPznnXDP2gwA+m4j+IBFdQmIbfpaZf2HPY11sscXOxBYoWOxptG8G8EMAfpyIHkIa088L370DwPdDGvV/CODvQNwCut0fIqLXiOgvZsrNbsvMDwH8CQDfCwn0+4qw/33sfwbwZaG8fxXAl/a4OIbOtWXM/DKAPwjgm0K5nwfgD+95nIstttgZGTEPKqKLLbbYiRkRfSMkmPGP3PaxLLbYYnfLFqVgscUWW2yxxRYDsEDBYosttthii52dEdFfCcnOfq7neyKiv0hEv0REP5vkRekvd3EfLLbYYosttth5GRH9s5BhxP89M3925vsvgiQv+yJIfNA3M3M2nsjaohQstthiiy222JkZM/9dAK8OrPLFEGBgZv5JAC8Q0SeNlbtAwWKLLbbYYovdPXsnTJIzAB8MywbtKBkN3/rsPX73254DaJ/cLHvYLe12lp2y1+bILiV+XB2gkP2LOLrdxH14Svf6XsfSbPzoYb5aOsSp0h43zqGqs9nHMLDfU/r5b8p+7o2XP8bMb7vt45hjv+/3/TZ+5ZU0g/iw/fRP/6P3A7gyi76dmb/9oAeWsaNAwbvf9hze901fCbgRIcJNuKWnrLPLuqdifodKau42s9f34+sY45nlb//Bx8J2szYT2+V63bbtcF/SHA1vxrrzyp153HvojmT29b6f+MT+9QYaVEfd73KnkCsj3TYHAN11MuVgfjl9ZfWtO7ZNs+3g14M2VvZtGLOc0Kf/2Lf+2i0fymx75ZWP4+/9/f921jZl8buvmPm37rHbD8FkPoVkPc1lMm3ZabsPbgsIyOX/jmGOdqh8j73+8W+L2UDg+TyBANjp2HcCpjM1mnF/8kDf2HP3u5u8ZW4KCIj4KECg5Z4iEADTzvtkjSGdrTl/+9sPQdKsExF9PoCPM/NHxjY6/wmR9sLhmY3f0Pr71uKO5tVgc9c/IdsJCA617x2KOpgXzPOs+5X9xFvU49Txfpbpz913qRQMdnUFMGgvN8JN2BAQDG83bz9n28ienfGhGvpoRPQ/AvhdAN4aJkT7BgArAGDmb4PMr/JFAH4JwGPIlO2jdjwo2Nd1sO/3fXasHr8td1dAOCYYnCtE7HHMhwqNyJWzMyiMtXhHtsmgcQI2k6HCNtRpUKeUk27HfPiQqKkqwbGBYAGBW7IDQwEzf/nI9wzgq+eWeztKwW0AwU3WhPsAgp7bGTX25Gh2XMEk26HMm0q7ofvZCw4m3Md3XS0Ycx0MXaZ9evynqhbsAgQLDJyBMW6uctrTzs99MFsfu+WaktxuysExGvtZyoI7ONlOthOGgaH9zgaEQ4PBHbW5YLCrWjDH0gZ2LJ5gTizBnPWmnNOhYWDX8jgT8/H02OHdB8eym4eCfVSAOU/1KdWieiy7qAZTGsgTUAsOZnMD8k7otHdSDw7ZWp2pWjDFDt2o37ZNdRv0ByHuto+pdgxVoa/MpwYWFijYwQ7x1J8SDKS2CxwcGgxOGSBmHNcpwUBqs/3RE1q8Q6oF56o89F2mqWrB2HZTttnFppR5SCDYpUG/TddCuu87CQk6+uAM7DhQcIykRZPQeIeablcQ2adh3dWlcJftNoFAf4oDNpR7xxzkyjzlxnyH4+qLJ2Cm3kaqL05zSozATakNY7/5Pg3woYDglOML7LHdHUBY3Ad52/WJHH0SZtZIh6gZbBm7AMIc1eAuqABDdlNAMHapx77foeGbDAeHarHuiAtBG4MhOBgDg2P1/PexfdwGQ7fHXYCBnN0ZQHjqlYJdrO+OPwQQHLt7sA8gTFUNzrXBH7NjAsGhn0Fb3syGd5JL4YbdCOdgY6rBXDA4hlqQBhm2vhsdRrgfEExp5M8NBPrsvM+DQWeiDp8OFOxipwAEQ/ub2uAd0p0wBR4mKw+3OALB2CwYuKnD3QEQDgUGd8FmZTEcUA32vVw3OTRx1wZ8VyA4VCO6i9qSyy751NsJ1KVT7DSgYBeVYAwITqFindO7nwIGd00tmHAuk4HgNp+3GTEJk9wJIy3dqFpwR1wIqfWpBunlug21QM3+rnNVgqlAcCwYOKSrpa+spxYWGGdTd98cFMxt+E8BCA7RU56TjOhQYHAO8HAoIDgl+J4JB/uAwUnZ7GzhIwmLBooccie01ttRAbAAccishp28BkcAgjkwcFuxFt38EWdyj+9tS6Dh8WwICGZNoDSxJutbb+4PPLWhXkYmADguEMy5vDv57yfCwT6Nzr6xBacemzB0CXNgMMZQQ2rBTWc3PDQQTIWBUwu6BJD8JncYEJZAw8QONavfvkBwyNn/0rKm/OBTVYMxMDiEEnCbasLIfkeBYOaztQ9j5bad3JhOkPEHwWAfteBEXQhzYgmA/tOYAgbHGo1g99sXZGj3M1fOnwoEU8o9RRDoszsPCGfS2btdpeBgMv9QFsQbqhl1P1Ph4NhgcION/qx5D24ICI75/NmyRwFh3xwIA2Bw6r39Q1nfJcwFII6BwTFtiuozphKkP/Uu6sBhwGf6usdIJHb3AGFxH+xuc1SC0ZTJt1BjToWDQ4DBHbNDAMFNXy7d3yQ46FnnGDPy3ZjNeMTmqgSpDcHBEBi0yjBqwU24EObEAEwBgkPCwOHiJfLLDwULze91rg/JednpQUHO5gLBXBiYUlnN7XVPgYN9e/PnEFA40fYFgttmp0m99l3B4JyCDg9gtvLPNXS5yzgEBnMb/H3dDH3bDrkU9gGCqcd60+CZ298+oHBOrpCOLaMPJtjk+T4PDASHzKp4iLH+Yw37PmrBXYGGgdOfdWn2AYcpIwqmqAYH9vXfdRdCXwM9BQymlHko3hprcPcBgl1h4BTVJz2mU5675Di2uA92s87TMQMIjgUDU8ocHSI4ohrs03jfgYZ/sII4BBAc4llMyxi43UbhoAcMztqNMGL7uA5UOegMZwuv9lJaMJirFhzahTAtEHB4m6lpkdvbjB/bbZs9xqcGEBYouEEbAoKbkF3n5CIYKqNv+2PFFpwAUBwNCI79/E0IIBzsxc8Fg126tPuoEpNG8+xY9o42BAe9l7kHDA4xEiE38iBXZp9KcEgg2AUEDpHx8BDzETwVgMAALVAw0yZVQpl1+oBgVs6C3qik6WXYcvq2O6ZicA42a5RC/1eDQHDTz90IHNx1eX+KzVUJGM36uZ57rkG3YDDVjdCUd/i+w9wGdwwI9oWBY80b0FfurrBwd90LfDYndTpQkFpakx4SCKbWADvHEUyAg7lgsKtacMKgscsz0nsJjjhUcVKjPhRE2AcGN6EW3LLtO+JAASGFgzlgcEy1ABhumKeoBLsAwTGnZz6E7Tu74Z2Eg0UpGLBjVmy7zra4S/mnkm74Fhv+WTkKplrPs7MrEOzreZmcl+CAYDDHdlEjBtc/QddBDg5y7oQpYDC2HwJHYJgT5zEEGIcCglOHgZztAwh3Bg4YCxTsZbuqBNn1bgBAdkkidJNqwTnZAYHLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1152x360 with 2 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"needs_background": "light"
|
||||
},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAE8CAYAAABQAcL1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9ebAtyV3f+f1VnXPf6/da3VpaLSG11JJAssBgGMCsYSM8YrEMASaYYTEzhpjxjGMAM2OWwICBcQwRDo+XgbDCjMAsxgximQE0RmGBxjFgbBZJDJsECCHcqFtSt7rVy+t+7757TuVv/silfpmVmZVVp+rcc+6tX8S955xasrK2zE9+f7/MJGbGYosttthiiy22WHXeGVhsscUWW2yxxQ7DFihYbLHFFltsscUALFCw2GKLLbbYYosZW6BgscUWW2yxxRYDsEDBYosttthiiy1mbIGCxRZbbLHFFlsMwAIFiy124YyI/hMRvfa887HYYosdny1QsNiFNSJ6WvwpIrolfv+NEen9v0T0386R10MwIjohop8xUMFE9JrzztNiiy22X1ugYLELa8x8p/0D8GcAvlAs+/Hzzt+B2q8C+CoAHzzvjCy22GL7twUKFrt0RkQVEX0rEf0JET1GRD9FRM81664S0b82y58gorcR0QuI6HsA/CUA/9woDf88km50X7Pua4joD4joBhG9l4j+e7Hfa4joQSL6FiJ6hIg+QERfTESvI6J3E9GHiejbxPbfbVr0P2nS+y0i+vih5xoaM58x8//GzL8KoNnpIi+22GJHaQsULHYZ7esBfDGAzwLwIgCPA3i9Wfc3AdwN4CUAngfgbwO4xczfDuDfA/g6ozR8XSTd6L5m3SMAvgDAXQC+BsA/I6JPFPu+EMBVAC8G8J0AfgC6xf5J0DDy94no5WL7LwLw0wCeC+D/APBzRLQeeK6LLbbYYp4tULDYZbS/DeDbmflBZr4N4LsBfCkRrQBsoCv0j2LmhpnfwcxPFaab3JeZf4GZ/4S1/TKAX4Su7OW+38PMGwBvBHAPgO9l5hvM/E4A7wIg1YB3MPPPmO3/KTRQfNrAc11sscUW82wpGBa7jHY/gJ8lIiWWNQBeAODHoFv6bySiZwP419CV6qYg3eS+RPRXAXwXgFdBw/g1AL8n9n2Mma1kb9WFh8X6WwDuFL/fZ78wsyKiB6GVgCHn+lDBOS222GKXyBalYLHLaO8D8FeZ+dni7yozP8TMG2b+n5n5YwB8BrTk/1+b/bJTiqb2JaIrAP5PAP8YwAuY+dkA3gyAdjiHl9gvRFQBuA/A+4ec6w7HXmyxxS6oLVCw2GW07wfwPUR0PwAQ0fOJ6IvM988moo8johrAU9Cyvm1lPwzgFalEM/ueALgC4EMAtkY1+Nwdz+GTiOhLjBvgfwRwG8CvDznXxDlcIaKr5ueJCZ7cBV4WW2yxI7IFCha7jPa9AN4E4BeJ6AZ0ZfqpZt0LAfwMdKX+BwB+GdotYPf7UiJ6nIi+L5JudF9mvgHg7wD4KehAv680x9/Ffh7Al5n0/isAX5JwceTONWZ/BO2qeDGAt5jv9++Y18UWW+xIjJiziuhiiy12YEZE3w0dzPhV552XxRZb7GLZohQstthiiy222GIAFihYbLHFFltssaMzIvohM9jZ7yfWExF9HxG9h4h+NxgXJZ3u4j5YbLHFFltsseMyIvrLAJ4G8K+Y+WMj618HPXjZ66DjiL6XmXPxRAAWpWCxxRZbbLHFjs6Y+VcAfDizyRdBAwMz868DeDYRfURfugsULLbYYostttjFsxdDDHIG4EGzLGuzjGj47PUd/BFX7x61b9SZMdDDMd4hMrY79sxHpOzPwuMMz+NcvdOvPqtnrp0djzu4W/0x9cIfcBsncQ0OTWKnQ+qdH3zsas92hanN6BodknJpNkrTLNoustHuV+MwXM1PNB98lJmff975GGKf93mfwo899uSgfd7xjne/E8CpWPQGZn7DpBmL2CxQ8KKrd+FHP7E7Xb3i8He3NOZICR3ux5H9VGdJPP2+Y01huQq4ou66mFxDYrsqks3wGGG64T5928fq0VheKZr/+Pmm6uZXfdZToNhJ6cQSy6MZ7B5zinRL1s1p4QNfuj7yEnBy28jy6LL47oPSdTtlVon9vuVHP6pklyJTPTVyanV42uF2nTIpc1y5LXvbhGkk9gmP7eWrewLx2xg/0T5gSe13nvZzH/6eB847D0PtsceexG/85v8+aJ9V/dmnzPzJOxz2IYiRT6FHPe0dyXRv7oO+cg7oVtKKxwGBYkoCAYPc3xCzeSk9j9QxYvmKlbvyPKPX4aiatztaIRCk90+kmQSIzDppROP+evNL/fmLLo9k8bzAJrSRdQv1/PVZReT+ouknbkl4KcNtutC9m3HhBUowWru+EAiY80CgwAcJBEdrDECpYX+725ugh1knIvo0AE8y8wf6djqqCZFKFYLovj2vbUlln9s2Vfba45LXCiCzj1iGbkHETNGWeTxP5KfH59fQPS+LVoApIIhZKQjsamEaudK5ovwDF66r0HkpqKJuyz6WbnRZN71DsvBu5N4WCwYx9YCov9Vcss0+rcQ9kgKCIduX5WXY9pdv4GyeqqJ3RkQ/AeA1AO4xE6J9F4A1ADDz90PPr/I6AO8BcBN6yvZeOzcoCFvMMZXAWz/SZZCDgSEg0GcyrVjdwqCOhB/CgT0fWY9JMAgr+liaKRuybcqmcB3EXBJT2KxAMHcJJtOPla4pALDrpnyQz8GSrogRZq9kHxyUgEEfD8116VOug6EqwRAgGAoDu8JRav8LDQsTQwEzf0XPegbwtUPTnQUK9iFtx1wGJfkofYn74hGsxSq5FCDEVAN7rD7VQKadqr/CdHI2ZNu9Wakzq6RFPwUQFJZQQyT63grQHjMFByUP8C5qwQWxPjhIqQZ9YJBTC0gcLwUeU1moEvQBwS4wsE91JDzWhYEExmHJTBnbi1LQV+4MVQn6gGAoDJQCQMm+YUVrj9sHBzkwyLkRcgrAebkQJn2RC05gtN98JAzs4qcP901CQqr2uSCyf85yl3cow8ikYrvGKu8hisEhcNUYIDg0GOjLw/HDwfTug7nsXNwHQyrhXYEgHSQ97ilj5IOKYvECMh85+b+09X6s8QKTKBNTqQTRLh3ptOcK2LPpRuEgpRqUgkHkWFNK9cWWOeTQ/KRVsv59ZUveT3MYGAyNLRgLDinXgVQJpgaCQwCBmB1qvgbZAgXjLNui72w7HAjSvRLKLbZtmKo8Ti4AMAcGY9SCfboFUvEE+7SiynoHICiGgb7tCipsYAAclIBBiVrQ2WeHpu85NpnDy58UYMxnuDrmTsi6CsQ6eclS4LEP2wUIhlS6u/ZKqC5TzylrjAUKSi0XfxALLrQ2FAjiMQfTWZiWPFonoDBQDUJ3QgkYzK0WdMcxKLta+w4w7B6oc+D8byCa6V4YGHrxC2utXjjYEQzOTS3Ys8nLHb2U5jMGB6lYgH14ZPqGisipBPC2C9MYDwNTd00M07sckLC4D5KWk+1LVYIcEEwBA0NdC7EKzy4J4aBUNUiBwaWynoq3U3H3AUE0kQFAMCWF9UBCEg5KwOBILAUnU1e+uc4bsZa9BINSN0KczyJuicjxYjZULpcVbcm+hzRokTzWhQWERSlobWx55Q3ek9tuABDEsrJLkGEqDVnxh3AwBAxillILkkAxk6IQcx3MrhLseiKdEWf836NhoDRfuZchUXNFW/Z9YHBIasHQw5h8joHgEk8J0L0NMdUgBwZTW+nARSmVIAcEc3ZLnMNsHi4eHDCIFyiYxHJxBKVAMBQGhrwasVRiwYYSDmLuhFi9MrVaMMVYBQdrQ90GJUAw9RDIJS6ESM0VVQ2OXDGYA0pi70esGM7BQQkY9AUdThlXEB+LRa5PH2nKbon6uMPPKjWSZO+xLqJ6sCgF/Zaq1FOxBCVA0KcOTB1bENtXqgJAFw5iqkFJq9+lscfYgpjNohIMJJ5BvQGGAsE+3AdhegW9CTot/FwTdggkTAEUsf0PgFHsY1UKB2PAQKaXuowllzgfI9B/Mf3eCv1A0AcDU4yzEEtjKChcCPWAcTTQfvBKQczGAEGJKyG2XYl1Km3zmYKDKcAgZlMpAWODDPvS2ZsNGJ2wFwimDjgEihWClGqQBINsrYTJHPW7tvLPI9BRMmdfZV4KBm77LJsNH8QoOwxxgdugDwhSMDDnYEup4wwBhENwb4w3XpSCmA2tcHPBhXpZfP0QGBiTp7BRm4opyMFBCRjkjt03L8I+uyYOAf+xgJE1eUNylXoOCErdB1OoBX0KQbg8qLWyMQFy2wwkHHIvhKkGqklW1OZTli/hZS9xAeTUgilcCLkqZAwQnDcM9B17rKvhKIyxQEGfpboiRuc4SLgN4uv7l4fr3LJEXku3c4VNRBkAuhAgl8t9LBjE1II4lIzpITey9V9Y1E0OJFP48IVlgWDO+IIUBITrYsGDYhuvUi+NhNvnKIeJ7GRBZOK89c07lYKDGBj0uRHmDEa0roOoh6ZAOQjX6bTymZ3qVIa8tTZPFxYOlkDDcuvrm2stF1io18s0qbNMLs8dx6VXoCLIlq9XuATHyqkDseUxMIjlb4qWd+4d7Et/MpUgpJ2p/fcio4OAYGq1IBdsGDZXe1SDJBgUqgWTWWH6+wSCmCUHiAyyMBQM9m22gi8BglIYmOtUwnRL3pyLCQeL+2AyK3Ub9AGBTCe8NSWVf8rCfUn0FgB89SCmDsTAoM2zDwY5tcDlpzCuYE73QizdwfCSkT/8St07cPx7CRCMjS0YCggj3AW53wfpBogB+wAgCJlmF8sNEAmkxyCQtyUFBuF+FhSmcCHYfIQqQcwFMBYIhuQt5XoYGvwnU+nb82LCweHbQUFBrnLOuQ1KgaAPBnbhOOnrt0amArfrYzEFqWV9FXYIBkNdCKn0c8cNXQexd7UUNFjxbPMLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1152x360 with 2 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"needs_background": "light"
|
||||
},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAE8CAYAAABQAcL1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9e7A1y1Uf9ls9e59zvu++H0JCD6QrJIVQYJ7F0yHgAMKYlDBFBUPA4IqdUAGME8AFAYNCTMpFJU6BwcYSAceALQMBLAfZAuyKMUQykiAIJMVCCJDuldDVvbr3fvd7nHP2nl75o7tn1vR09/TMntmvM79b3z17z/T09Mzux69/a/VqYmbMmDFjxowZM2aoXRdgxowZM2bMmLEfmEnBjBkzZsyYMQPATApmzJgxY8aMGRYzKZgxY8aMGTNmAJhJwYwZM2bMmDHDYiYFM2bMmDFjxgwAMymYMePoQER/TERfuOtyzJgx4/Awk4IZRwsiuin+aSK6I77/lwPy+7+J6K9OUdZ9ABF9FhH9KhF9hIg+TEQ/R0QfvetyzZgxY3uYScGMowUz3+3+AXgfgP9cHPuZXZdvD/EAgNcAeAmAFwN4FsBP7rJAM2bM2C5mUjDjyoGIFBF9JxH9IRE9SUQ/S0QP2nNnRPTT9vjTRPQWInouEf0AgP8EwI9YpeFHAvkGr7Xn/goRvYuIniWi9xLRfyOu+3wiepSI/iYRPU5EHySiLyeiLyWid9uZ+/8g0r+aiH6eiP6Zze+3ieiT+j6rD2b+l8z8c8x8g5lvA/gRAJ+7ybueMWPGYWEmBTOuIr4FwJcD+E8BPB/AUwB+1J77egD3AXgRgIcAfCOAO8z83QD+HYBvtkrDNwfyDV5rzz0O4MsA3AvgrwD434joU8W1zwNwBuAFAL4XwGsBfC2AT4MhI3+LiB4R6V8F4OcAPAjgnwD4JSJa9nzWLnwegHdkpp0xY8YRYCYFM64ivhHAdzPzo8x8AeDVAL6SiBYAVjAD+suYuWTmtzHzjcx8o9cy8y8z8x+ywb8F8Cswg7289geYeQXgdQAeBvBDzPwsM78DwDsBSDXgbcz88zb934UhFJ/V81mjIKI/A0NOviPz2WfMmHEESHYMM2YcKV4M4BeJSItjJYDnAvgpmJn+64jofgA/DTOorjLyjV5LRH8ewPcBeAUMGb8O4PfEtU8yc2k/O3XhQ+L8HQB3i+/vdx+YWRPRozBKQJ9nfSz0EET0MgD/EsC3MvO/SzzvjBkzjgyzUjDjKuL9AP48M98v/p0x82PMvGLm/5GZPx7A58BI/n/ZXpfcUjR2LRGdAvg/AfwvAJ7LzPcDeAMA2uAZXuQ+EJEC8EIAH+jzrKFMiejFAH4NwP/EzD+1QflmzJhxgJhJwYyriB8D8AN2AAQRPYeIXmU/fwERfSIRFQBuwMj6bpb9IQAvjWWauPYEwCmADwNYW9Xgizd8hk8joq+wZoC/AeACwJv7PGug/C8A8G8A/Agz/9iG5ZsxY8YBYiYFM64ifgjA6wH8ChE9CzOYfqY99zwAPw8zqL8LwL+FMQu4676SiJ4ioh8O5Bu8lpmfBfDXAfwsjKPf19j7b4J/DuCrbH5fB+ArIiaO1LP6+KswpOfVMsbDhuWcMWPGAYGYk4rojBkz9gxE9GoYZ8av3XVZZsyYcVyYlYIZM2bMmDFjBoCZFMyYMWPGjBkHByL6CRvs7Pcj54mIfpiI3kNEb/fiosTznc0HM2bMmDFjxmGBiD4PwE0A/5iZPyFw/kthgpd9KYwf0Q8xc8yfqMKsFMyYMWPGjBkHBmb+dQAfSSR5FQxhYGZ+M4D7czY4m0nBjBkzZsyYcXx4AUSQMwCP2mNJTBLR8J7FdX7Oyf1TZA2gI4LMFvPowiaRaca4Pvs+G98o/20SgPvuvezxcCO9hW29zG1ilEqckcmWLYxPPXM6Wd5DH2XIda1rJn7VV8UQ/Oj5B59g5ufsuhx98MpXfgY/+eQzva5529ve/Q4A5+LQa5j5NaMWLIBJSMFzTu7H//xxZtv5VEXViZMc6cVT1+j4KZPnjlpN16CbkmtU5FqKvNl4+tQ94i8mll9OWWL5/YVXvi9Lo6K8m+fddIx7bQmcquQ+utLmDEQ59+tqXFW6YY2MRf4//8ZH4glD1w5gfH37kVDfoQP39fPlxLnQ9f59/LLE+rDYz5PzcxwKmXBv69ve9eo/2WlBBuDJJ5/Bv/+tf9jrmkXxBefM/Okb3PYxiMinMFFPg5FMJfbSfDA2IWAejxBoULAxp9B1/9z+dh/RlxCMfPNRbrpPhAAw5ckuU1e6/Xq0IHjDBkDg6l8uFMVfXahTDBF71XM47fqphvZRQwgBi3+HgkMqawsMQOt+/zbH62HCrBMRfRaAZ5j5g10X7WxDpL4Tik0IQVb+PXvPUPquToI5rhpohDsjzeHOhEHBTjCW/kqi40UMIgOb0ugebZ0U9VMNZjTaRI6CoCjctyi0fyqi7aqNOSrBUEKwCTTH32tKdRwDh9saeKyBvgIR/VMAnw/gYbsh2vcBWAKADVP+BpiVB+8BcBtmy/ZO7N0uiX2lwNhrzmm8fYlAn/xiBMGVK0QO+hKDMTB1I67v0y9954Ddld/YhGAsTU3mk9FHuHImyUFsZKsywd72ppuqBCnkEoQ+xKCdhrP7kbH5XahsY5KB1OA/5Jpt9TV7i5FJATN/dcd5BvBNffOdlBSMVQViFX0oIRibDKTukSIHfYhBMI+IWrAt5N5735SLfGl+2nJU+WeSg42IwSZ556JnHlMSAh+ursbIQe7r61ILNvgZGuhSCaYiBEOIQC5k3leOIDB259TWE3unFORiCCHobSKI5NVnkNOgUYjBpmoBo795ed8G82wkCp5FCLbtaZNJDq6KOSG33g15FSlyEBrQc9SCXKQIw1TjRU62UxKBnHteDYIwvvlgKuyEFEQbRoY3bwpxz9zuSr+pw3eqI0upBmMoBrvATlWC9FKKzfLe5UvPGIGSxCA16uyxCWEoYj911owfPJgY+GpBHxPCEAxVCbpewy7IQAiuHEdPDmZSMB1ylwyZtPGKP+akq9WRBAf6sGqQckD07+HnGzIh7Iuz4T6UQSKpEvQlA30fLreyZagGx6AYTGk6kD9N2tUirBpsqhhsakJI3WcMQrAvZMCHZjpeYuBWHxwADo4UjEEIttGfunv4Y0dMNQgRg12qBfs2oGdhqNkg9yVv8lJaFaGjEnaMQlFiMJZRuy/2lKS4195FDnKIQeOaTLVgU4Gm0z+qByEYQgb6ln3TbuN4zQqz+aA3NjUdtK4dSAiG3DLVEGKz9pBqkKMY7IMKkGM6mKyMA/LdiBBMt+zD/E06D2I8gzYwfIQauS/bpoOhQ5d6kEMMuswIfTDE3OC/tlxCkEsGxhiCQ3kMbUFHZVaYlYJpkKMSDCEDm1Y5eX3o7inVoIsY5KgFu16FsBcYO+bAtphXFzlImBP2Ti2YEPLXYHFsyFPGXk/InLCNVyn7MB057mMMQrCtGiJ/ryE4DrMCg3bBhgfgoEiBjxxCEHdqnAapBhCa5Q8hBrtUC7ahEowZYTCaV4wQ9I5fkPIw7VHLukafiGqwF/4FPe6f0y92/QIU+dy6V+Jciov5qoH8aVJqwZQOh/KeOYRgH8hA7L5D3tBREINZKdjMntaOE95EVnCiAR65setiiI0JsQaQSwx2iV2bJ1jzKMRgEkLQp1xBu1Fq4M9QDbbl7XYECKkLPnKsOK1r0P0zbOv191EH9qU2dCmrMRw0MWAcTHs8aKVAwmfo/RxwNrhvx6qDEDnIIQa5KxI2Qd8GtlNfgrFuOoQQjPlQOa7xPUeUXmrBDpcm7lI97SIIPjnwTQljDPJd18dMB10qQfMe7bqaW+xNnm9oE2H0JwaHidnRcCN0OgN2mA1SO5Xl3iOUd85sPuY/4Ff+ULoUMegyIfh+BbswMexTXKCgShC6+bbIQCr/VOCLKaPpbBH7ZE51v2qMHMhXLk0JMTNCrglhCLFIEYIuk0HqVqOHXO6YHKWwqc/BQWB2NOyHrv0Ocp1vgJyGk7pPVzni533CEBr0c1SDXZsSho6Dva8bst4ydI/AjbMJQQwZD9PXvNEZohiIk4MMYhBUC0aZ3nad785/U0KwqWQcm13GyEHqtaX8C4Yg1J/lmEZlkj7qwLYU7Maqjcym0lc1ODjsEzNOYC9IQQqdfZKoRkMIwViOQTFFIUYOuswJVdqEWrBNpEwHu/ZBGIwgeYg/zCZ+DvLa3hEJp46/u0/Q4y9Bk/mFBtAQOZCvPMeUMMUOijGVIEUIxiADfSO25uaXk8fxqgaz+WArSA3oXYQgdu2Qht0OOmQ7j4Scn2LFU6gFmzSysQjBziII55gNIg8y5koIP89eywkzZv1bX4mwBZVgbKQIgk8OUqaEKj/EuZm7PseNw+UR3B55ICHo9j/oKFRm2twm0pccHB8xOAwcFCnIXbfLkeNAmwyMwe5lHs2lhE1ykCIGKTNCdH+EDfwGQrOxKWf8O/Ul8DEWIch9qMioESUHuS7xe64WTE0Icn7q9GDm2mWbHMSIARLHnVowxtLEYEwWeV6UOddnaiq+2NdUkNtvHRUxYGzPdrMhJicFOSx5CGJmgxghCDXSICvvWQ5/XHB5+uRgCDEIYdsmhE1VgmRZt/Ewfv4ZhKD3Usa+ZQj4AwARcuD3uH07lk38CpJB+Cdo1Yn7jenr0n7NbXIQIwYhx8MpuVl4cI8Tgm2SgRBy1YDcdMdDDGbzwc6RaghjkIHQdbLv98nBEGKwTafDMWP2VGmGFWWn2NhJMQeRKIVZzoKt78189iKYEQaoBIH0UylXsVWhPjnIIQabwPURvukgpRL0IQTJPjCzjDF0PX2uepAzEToaYjCTgmFozu5rNNftdqsEobQ5W5CG0qXgRx50cP1+01GwNif0sa/5+WwLMZXgYAhBT5WgRQjGCoPcIxjLine truncated
|
||||
"text/plain": [
|
||||
"<Figure size 1152x360 with 2 Axes>"
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"needs_background": "light"
|
||||
},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if not os.path.isfile('data-airfoils-test.npz'):\n",
|
||||
" import urllib.request\n",
|
||||
" url=\"https://physicsbaseddeeplearning.org/data/data_test.npz\"\n",
|
||||
" print(\"Downloading test data, this should be fast...\")\n",
|
||||
" urllib.request.urlretrieve(url, 'data-airfoils-test.npz')\n",
|
||||
"\n",
|
||||
"nptfile=np.load('data-airfoils-test.npz')\n",
|
||||
"print(\"Loaded {}/{} test samples\\n\".format(len(nptfile[\"test_inputs\"]),len(nptfile[\"test_targets\"])))\n",
|
||||
"\n",
|
||||
"testdata = DfpDataset(nptfile[\"test_inputs\"],nptfile[\"test_targets\"])\n",
|
||||
"testLoader = torch.utils.data.DataLoader(testdata, batch_size=1, shuffle=False, drop_last=True) \n",
|
||||
"loader_test = Dataloader( \"airfoils-test\", batch_size=1, normalize_data=None, shuffle=False )\n",
|
||||
"loss = nn.L1Loss()\n",
|
||||
"\n",
|
||||
"net.eval()\n",
|
||||
"L1t_accum = 0.\n",
|
||||
"for i, validata in enumerate(testLoader, 0):\n",
|
||||
" inputs_curr, targets_curr = validata\n",
|
||||
" inputs.data.copy_(inputs_curr.float())\n",
|
||||
" targets.data.copy_(targets_curr.float())\n",
|
||||
"for i, testdata in enumerate(loader_test, 0):\n",
|
||||
" inputs_curr, targets_curr = testdata\n",
|
||||
" inputs = inputs_curr.float()\n",
|
||||
" targets = targets_curr.float()\n",
|
||||
"\n",
|
||||
" outputs = net(inputs)\n",
|
||||
" \n",
|
||||
" outputs_curr = outputs.data.cpu().numpy()\n",
|
||||
" inputs_curr = inputs_curr.cpu()\n",
|
||||
" targets_curr = targets_curr.cpu()\n",
|
||||
" \n",
|
||||
" L1t_accum += loss(outputs, targets).item()\n",
|
||||
" if i<3: plot(targets_curr[0] , outputs_curr[0], mask=inputs_curr[0][2], title=\"Test sample %d\"%(i))\n",
|
||||
"\n",
|
||||
" lossL1t = criterionL1(outputs, targets)\n",
|
||||
" L1t_accum += lossL1t.item()\n",
|
||||
" if i<3: showSbs(targets_curr[0] , outputs_curr[0], title=\"Test sample %d\"%(i))\n",
|
||||
"\n",
|
||||
"print(\"\\nAverage test error: {}\".format( L1t_accum/len(testLoader) ))"
|
||||
"print(\"\\nAverage relative test error: {}\".format( L1t_accum/len(loader_test) ))"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -746,11 +598,11 @@
|
||||
"id": "By722sgwFbwG"
|
||||
},
|
||||
"source": [
|
||||
"The average test error with the default settings should be ca. 0.03. As the inputs are normalized, this means the average error across all three fields is 3% w.r.t. the maxima of each quantity. This is not too bad for new shapes, but clearly leaves room for improvement.\n",
|
||||
"The average test error with the default settings should be below 0.025. As the inputs are normalized, this means the average relative error across all three fields is around 2.5% w.r.t. the maxima of each quantity. This is not too bad for new shapes, but clearly leaves room for improvement.\n",
|
||||
"\n",
|
||||
"Looking at the visualizations, you'll notice that especially high-pressure peaks and pockets of larger y-velocities are missing in the outputs. This is primarily caused by the small network, which does not have enough resources to reconstruct details.\n",
|
||||
"\n",
|
||||
"Nonetheless, we have successfully replaced a fairly sophisticated RANS solver with a very small and fast neural network architecture. It has GPU support \"out-of-the-box\" (via pytorch), is differentiable, and introduces an error of only a few per-cent. With additional changes and more data, this setup can be made highly accurate {cite}`chen2021highacc`. \n",
|
||||
"Nonetheless, we have successfully replaced a fairly sophisticated RANS solver with a small and fast neural network architecture. It has GPU support \"out-of-the-box\" (via pytorch), is differentiable, and introduces an error of only a few per-cent. With additional changes and more data, this setup can be made highly accurate {cite}`chen2021highacc`.\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
@@ -765,7 +617,7 @@
|
||||
"source": [
|
||||
"## Next steps\n",
|
||||
"\n",
|
||||
"There are many obvious things to try here (see the suggestions below), e.g. longer training, larger data sets, larger networks etc. \n",
|
||||
"There are many obvious things to try here (see the suggestions below), e.g. longer training, larger data sets, larger networks etc.\n",
|
||||
"\n",
|
||||
"* Experiment with learning rate, dropout, and network size to reduce the error on the test set. How small can you make it with the given training data?\n",
|
||||
"\n",
|
||||
@@ -777,12 +629,14 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"name": "supervised-airfoils.ipynb",
|
||||
"gpuType": "A100",
|
||||
"machine_shape": "hm",
|
||||
"provenance": []
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
@@ -796,9 +650,9 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.12.4"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Reference in new issue
Block a user