daviddoji/pbdl-book
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update SoL code

Browse Source
This commit is contained in:
NT 2021-06-06 15:37:24 +02:00
parent 19b4efcf40
commit f939b89ecd
2 changed files with 65 additions and 95 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

142
diffphys-code-sol.ipynb
View File

@ -8,9 +8,14 @@
"source": [
"# Reducing Numerical Errors with Deep Learning\n",
"\n",
"First, we'll target numerical errors that arise in the discretization of a continuous PDE $\\mathcal P^*$, i.e. when we formulate $\\mathcal P$. This approach will demonstrate that, despite the lack of closed-form descriptions, discretization errors often are functions with regular and repeating structures and, thus, can be learned by a neural network. Once the network is trained, it can be evaluated locally to improve the solution of a PDE-solver, i.e., to reduce its numerical error. The resulting method is a hybrid one: it will always run (a coarse) PDE solver, and then improve if at runtime with corrections inferred by an NN.\n",
"First, we'll target numerical errors that arise in the discretization of a continuous PDE $\\mathcal P^*$, i.e. when we formulate $\\mathcal P$. This approach will demonstrate that, despite the lack of closed-form descriptions, discretization errors often are functions with regular and repeating structures and, thus, can be learned by a neural network. Once the network is trained, it can be evaluated locally to improve the solution of a PDE-solver, i.e., to reduce its numerical error. The resulting method is a hybrid one: it will always run (a coarse) PDE solver, and then improve it at runtime with corrections inferred by an NN.\n",
"\n",
" \n",
"Pretty much all numerical methods contain some form of iterative process. That can be repeated updates over time for explicit solvers,or within a single update step for implicit solvers. Below we'll target iterations over time, an example for the second case could be found [here](https://github.com/tum-pbs/CG-Solver-in-the-Loop).\n",
"Pretty much all numerical methods contain some form of iterative process: repeated updates over time for explicit solvers, or within a single update step for implicit solvers. \n",
"An example for the second case could be found [here](https://github.com/tum-pbs/CG-Solver-in-the-Loop),\n",
"but below we'll target the first case, i.e. iterations over time.\n",
"[[run in colab]](https://colab.research.google.com/github/tum-pbs/pbdl-book/blob/main/diffphys-code-sol.ipynb)\n",
"\n",
"\n",
"## Problem formulation\n",
"\n",
@ -47,7 +52,11 @@
"\\newcommand{\\vr}[1]{\\mathbf{r}_{#1}} \n",
"\\newcommand{\\vcN}{\\vs} \n",
"\\newcommand{\\project}{\\mathcal{T}} \n",
"\\vc{t+n} = \\pdec(\\pdec(\\cdots \\pdec( \\mathcal{T} \\vr{t} )\\cdots)) = \\pdec^n ( \\mathcal{T} \\vr{t} ) .\n",
"\\pdec^n ( \\mathcal{T} \\vr{t} ) = \\pdec(\\pdec(\\cdots \\pdec( \\mathcal{T} \\vr{t} )\\cdots)) .\n",
"$ \n",
"The corresponding state of the simulation is\n",
"$\n",
"\\mathbf{s}_{t+n} = \\mathcal{P}^n ( \\mathcal{T} \\mathbf{r}_{t} ) .\n",
"$\n",
"Here we assume a mapping operator $\\mathcal{T}$ exists that transfers a reference solution to the source manifold. This could, e.g., be a simple downsampling operation.\n",
"Especially for longer sequences, i.e. larger $n$, the source state \n",
@ -56,7 +65,7 @@
"$\\newcommand{\\vr}[1]{\\mathbf{r}_{#1}} \\vr{t+n}$. \n",
"This is what we will address with an NN in the following.\n",
"\n",
"We'll use an $L^2$-norm in the following to quantify the deviations, i.e., \n",
"As before, we'll use an $L^2$-norm to quantify the deviations, i.e., \n",
"an error function $\\newcommand{\\loss}{e} \n",
"\\newcommand{\\corr}{\\mathcal{C}} \n",
"\\newcommand{\\vc}[1]{\\mathbf{s}_{#1}} \n",
@ -77,13 +86,14 @@
"To distinguish the original states $\\mathbf{s}$ from the corrected ones, we'll denote the latter with an added tilde $\\tilde{\\mathbf{s}}$.\n",
"The overall learning goal now becomes\n",
"\n",
"$\n",
"$$\n",
"\\newcommand{\\corr}{\\mathcal{C}} \n",
"\\newcommand{\\vr}[1]{\\mathbf{r}_{#1}} \n",
"\\text{argmin}_\\theta | ( \\mathcal{P}_{s} \\corr )^n ( \\mathcal{T} \\vr{t} ) - \\mathcal{T} \\vr{t}|^2\n",
"$\n",
"$$\n",
"\n",
"A crucial bit here that's easy to overlook is that the correction depends on the modified states, i.e.\n",
"To simplify the notation, we've dropped the sum over different samples here (the $i$ from previous versions).\n",
"A crucial bit that's easy to overlook in the equation above, is that the correction depends on the modified states, i.e.\n",
"it is a function of\n",
"$\\tilde{\\mathbf{s}}$, so we have \n",
"$\\newcommand{\\vctN}{\\tilde{\\mathbf{s}}} \\newcommand{\\corr}{\\mathcal{C}} \\corr (\\vctN | \\theta)$.\n",
@ -193,7 +203,7 @@
"Domain setup for the wake flow case.\n",
"```\n",
"\n",
"The solver applies inflow boundary conditions for the y-velocity with a pre-multiplied mask (`velBCy,velBCyMask`), and then calls `super().step()` to run the _regular_ phiflow fluid solving step.\n"
"The solver applies inflow boundary conditions for the y-velocity with a pre-multiplied mask (`velBCy,velBCyMask`), to set the y components at the bottom of the domai. It then calls `super().step()` to run the _regular_ phiflow fluid solving step.\n"
]
},
{
@ -234,15 +244,13 @@
"source": [
"## Network architecture\n",
"\n",
"We'll also define two alternative neural networks to represent \n",
"$\\newcommand{\\vcN}{\\mathbf{s}} \\newcommand{\\corr}{\\mathcal{C}} \\corr$: \n",
"\n",
"In all cases we'll use fully convolutional networks, i.e. networks without any fully-connected layers. The\n",
"inputs are: \n",
"We'll also define two alternative versions of a neural networks to represent \n",
"$\\newcommand{\\vcN}{\\mathbf{s}} \\newcommand{\\corr}{\\mathcal{C}} \\corr$. In both cases we'll use fully convolutional networks, i.e. networks without any fully-connected layers. We'll use Keras within tensorflow to define the layers of the network (mostly via `Conv2D`), typically activated via ReLU and LeakyReLU functions, respectively.\n",
"The inputs to the network are: \n",
"- 2 fields with x,y velocity\n",
"- plus the Reynolds number as constant channel.\n",
"\n",
"The output is : \n",
"The output is: \n",
"- a 2 component field containing the x,y velocity\n",
"\n",
"First, let's define a minimal network consisting only of three convolutional layers with ReLU activations (we're also using keras here for simplicity). The input channel dimension is defined via the `tensor_in`, then we'll go to 32 and 64 features, before reducing to 2 channels in the output. "
@ -329,9 +337,11 @@
"source": [
"Next, we're coming to two functions which are pretty important: they transform the simulation state into an input tensor for the network, and vice versa. Hence, they're the interface between _keras/tensorflow_ and _phiflow_.\n",
"\n",
"The `to_keras` function uses the `staggered_tensor` from phiflow (a 2 component tensor instead of 2 separate grids), from which we discard the outermost layer. We then add a constant channel via `tf.constant` that is multiplied by the desired Reynolds number.\n",
"The `to_keras` function uses the `staggered_tensor` from phiflow (a 2 component tensor instead of 2 separate grids), from which we discard the outermost layer. We then add a constant channel via `tf.constant` that is multiplied by the desired Reynolds number. The resulting stack of grids is provided as a tensor as input to the neural network. \n",
"\n",
"After network evaluation, we transform the output tensor back into a staggered velocity grid for phiflow. (Note: these are two _centered_ grids with different sizes, so we leave the work to the`unstack_staggered_tensor` function in `StaggeredGrid()` constructor)."
"After network evaluation, we transform the output tensor back into a phiflow grid via the `to_staggered` function. \n",
"It converts the 2-component tensor that is returned by the network into a phiflow staggered grid object, so that it is compatible with the velocity field of the fluid simulation.\n",
"(Note: these are two _centered_ grids with different sizes, so we leave the work to the`unstack_staggered_tensor` function in `StaggeredGrid()` constructor)."
]
},
{
@ -370,7 +380,7 @@
"\n",
"We actually have a lot of data dimensions: multiple simulations, with many time steps, each with different fields. This makes the code below a bit more difficult to read.\n",
"\n",
"The data format for `dataPreloaded`: is `['sim_name', frame, field (dens & vel)]`, where each field has dimension `[batch-size, y-size, x-size, channels]` (this is the standard phiflow export)."
"The data format for the numpy array `dataPreloaded`: is `['sim_name', frame, field (dens & vel)]`, where each field has dimension `[batch-size, y-size, x-size, channels]` (this is the standard for a phiflow export)."
]
},
{
@ -460,7 +470,7 @@
"id": "twIMJ3V0N1FX"
},
"source": [
"The `nextEpoch`, `nextBatch`, and `nextStep` functions will be called at training time.\n",
"The `nextEpoch`, `nextBatch`, and `nextStep` functions will be called at training time to randomize the training data.\n",
"\n",
"Now we need one more function that compiles the data for a mini batch to train with, called `getData` below. It returns batches of the desired size in terms of marker density, velocity, and Reynolds number.\n"
]
@ -518,7 +528,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@ -526,15 +536,7 @@
"id": "59EBdEdj0QR2",
"outputId": "8043f090-4e7b-4178-d2d2-513981e3905b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Data stats: {'std': (2.2194703, (0.32598782, 0.1820292)), 'ext.std': [1732512.6262166172]}\n"
]
}
],
"outputs": [],
"source": [
"nsims = 6\n",
"batch_size = 3\n",
@ -549,11 +551,11 @@
"id": "0N92RooWPzeA"
},
"source": [
"Additionally, we've defined several global variables to control the training and the simulation.\n",
"Additionally, we've defined several global variables to control the training and the simulation in the next code cells.\n",
"\n",
"The most important and interesting one is `msteps`. It defines the number of simulation steps that are unrolled at each training iteration. This directly influences the runtime of each training step, as we first have to simulation all steps forward, and then backpropagate the gradient through all `msteps` simulation steps interleaved with the NN evaluations. However, this is where we'll receive important feedback in terms of gradients how the inferred corrections actually influence a running simulation. Hence, larger `msteps` are typically better.\n",
"The most important and interesting one is `msteps`. It defines the number of simulation steps that are unrolled at each training iteration. This directly influences the runtime of each training step, as we first have to simulate all steps forward, and then backpropagate the gradient through all `msteps` simulation steps interleaved with the NN evaluations. However, this is where we'll receive important feedback in terms of gradients how the inferred corrections actually influence a running simulation. Hence, larger `msteps` are typically better.\n",
"\n",
"In addition we define the `source` and `reference` simulations below (note, the reference is just a placeholder for data, only the `source` simulation is actually executed). We also define the actual NN `network`. All three are initialized with the size given in the data set (`dataset.resolution`)."
"In addition we define the `source` and `reference` simulations below. The reference is just a placeholder for data, only the `source` simulation is actually executed. We also define the actual NN `network`. All three are initialized with the size given in the data set `dataset.resolution`."
]
},
{
@ -639,9 +641,9 @@
"source": [
"## Interleaving simulation and NN\n",
"\n",
"Now comes the **most crucial** step in the whole setup: we define the chain of simulation steps and network evaluations to be used at training time. After all the work defining helper functions, it's acutally pretty simple: we loop over `msteps`, call the simulator via `KarmanFlow.step` for an input state, and afterwards evaluate the correction via `network(to_keras())`. The correction is then added to the last simulation state in the `prediction` list (we're actually simply overwriting the last simulated step `prediction[-1]` with `velocity + correction[-1]`.\n",
"Now comes the **most crucial** step in the whole setup: we define the chain of simulation steps and network evaluations to be used at training time. After all the work defining helper functions, it's acutally pretty simple: we loop over `msteps`, call the simulator via `KarmanFlow.step` for an input state, and afterwards evaluate the correction via `network(to_keras(...))`. The NN correction is then added to the last simulation state in the `prediction` list (we're actually simply overwriting the last simulated step `prediction[-1]` with `velocity + correction[-1]`.\n",
"\n",
"One other important thing that's happening here is normalization: the inputs to the network are divided by the standard deviations in `dataset.dataStats`. This is slightly complicated as we have to append the scaling for the Reynolds numbers to the normalization for the velocity. After evaluating the `network`, we only have a velocity left, so we can simply multiply by the standard deviation again (`* dataset.dataStats['std'][1]`)."
"One other important thing that's happening here is normalization: the inputs to the network are divided by the standard deviations in `dataset.dataStats`. This is slightly complicated as we have to append the scaling for the Reynolds numbers to the normalization for the velocity. After evaluating the `network`, we only have a velocity left, so we can simply multiply by the standard deviation of the velocity again (via `* dataset.dataStats['std'][1]`)."
]
},
{
@ -715,7 +717,7 @@
"source": [
"## Training\n",
"\n",
"For the training, we use a standard Adam optimizer, and only run 4 epochs by default. This could (should) be increased for the larger network or to obtain more accurate results."
"For the training, we use a standard Adam optimizer, and only run 4 epochs by default. This should be increased for the larger network or to obtain more accurate results."
]
},
{
@ -728,14 +730,14 @@
"source": [
"lr = 1e-4\n",
"adapt_lr = True\n",
"resume = 0 # load existing network?\n",
"resume = 0 # set to 1 to load existing network\n",
"epochs = 4\n",
"\n",
"opt = tf.compat.v1.train.AdamOptimizer(learning_rate=lr_in)\n",
"train_step = opt.minimize(loss)\n",
"\n",
"tf_session = tf.Session() \n",
"scene = Scene.create(\".\", count=batch_size, mkdir=False, copy_calling_script=False)\n",
"scene = Scene.create(\".\", count=batch_size, mkdir=False, copy_calling_script=False) # not really used here\n",
"sess = Session(scene, session=tf_session)\n",
"tf.compat.v1.keras.backend.set_session(tf_session)\n",
"\n",
@ -753,7 +755,7 @@
"id": "p8hUXJDkRQST"
},
"source": [
"As this setups supports an (optional) fairly accurate training with the medium sized network from above, we'll define helper function for scheduling learning rate decay. This helps to make the network converge later on in the training. The steps below (after 10,15 etc. epochs) were determined heuristically from previous runs. Feel free to experiment."
"As this setups supports an (optional) fairly accurate training with the medium sized network from above, we'll define a helper function for scheduling learning rate decay. This helps to make the network converge later on in the training. The steps below (after 10,15 etc. epochs) were determined heuristically from previous runs, and are not active by the default run with `epochs=4` (feel free to change and experiment to train more accurate networks)."
]
},
{
@ -779,14 +781,14 @@
"id": "lrALctV1RWBO"
},
"source": [
"Finally, we can start training! This is very straight forward now, we simply loop over the desired number of iterations, get a batch each time via `getData`, feed it into the source simulation input `source_in`, and compare it in the loss with the `reference` data for the batch.\n",
"Finally, we can start training the NN! This is very straight forward now, we simply loop over the desired number of iterations, get a batch each time via `getData`, feed it into the source simulation input `source_in`, and compare it in the loss with the `reference` data for the batch.\n",
"\n",
"The setup above will automatically take care that the differentiable physics setup used here provides the right gradient information, and connects to the tensorflow network. Be warned: due to the complexity of the setup, this training run can take a while... (If you have a saved `final.h5` model from a previous run, you can potentially skip this block and load the previously trained model instead.)"
"The setup above will automatically take care that the differentiable physics solver used here provides the right gradient information, and provides it to the tensorflow network. Be warned: due to the complexity of the setup, this training run can take a while... (If you have a saved `final.h5` model from a previous run, you can potentially skip this block and load the previously trained model instead.)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
@ -794,47 +796,7 @@
"id": "m3Nd8YyHRVFQ",
"outputId": "148d951b-7070-4a95-c6d7-0fd91d29606e"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epoch 001/004, batch 001/002, step 0001/0496: loss=6816.91162109375\n",
"epoch 001/004, batch 001/002, step 0002/0496: loss=4036.171875\n",
"epoch 001/004, batch 001/002, step 0003/0496: loss=1627.97412109375\n",
"epoch 001/004, batch 001/002, step 0032/0496: loss=344.7778625488281\n",
"epoch 001/004, batch 001/002, step 0063/0496: loss=166.46800231933594\n",
"epoch 001/004, batch 001/002, step 0094/0496: loss=113.92779541015625\n",
"epoch 001/004, batch 001/002, step 0125/0496: loss=91.14945983886719\n",
"epoch 001/004, batch 001/002, step 0156/0496: loss=73.96617126464844\n",
"epoch 001/004, batch 001/002, step 0187/0496: loss=60.41331481933594\n",
"epoch 001/004, batch 001/002, step 0218/0496: loss=60.62569046020508\n",
"epoch 001/004, batch 001/002, step 0249/0496: loss=54.98934555053711\n",
"epoch 001/004, batch 001/002, step 0280/0496: loss=46.72457504272461\n",
"epoch 001/004, batch 001/002, step 0311/0496: loss=41.06715393066406\n",
"epoch 001/004, batch 001/002, step 0342/0496: loss=41.29859924316406\n",
"epoch 001/004, batch 001/002, step 0373/0496: loss=37.749908447265625\n",
"epoch 001/004, batch 001/002, step 0404/0496: loss=27.011524200439453\n",
"epoch 001/004, batch 001/002, step 0435/0496: loss=28.442049026489258\n",
"epoch 001/004, batch 001/002, step 0466/0496: loss=31.86280059814453\n",
"epoch 001/004, batch 002/002, step 0001/0496: loss=30.664321899414062\n",
"epoch 001/004, batch 002/002, step 0002/0496: loss=29.717803955078125\n",
"epoch 001/004, batch 002/002, step 0003/0496: loss=29.26577377319336\n",
"epoch 002/004, batch 001/002, step 0001/0496: loss=17.874845504760742\n",
"epoch 002/004, batch 001/002, step 0125/0496: loss=18.53813362121582\n",
"epoch 002/004, batch 001/002, step 0249/0496: loss=21.788429260253906\n",
"epoch 002/004, batch 001/002, step 0373/0496: loss=8.640719413757324\n",
"epoch 003/004, batch 001/002, step 0001/0496: loss=6.8903045654296875\n",
"epoch 003/004, batch 001/002, step 0125/0496: loss=8.793163299560547\n",
"epoch 003/004, batch 001/002, step 0249/0496: loss=5.2972636222839355\n",
"epoch 003/004, batch 001/002, step 0373/0496: loss=6.923160076141357\n",
"epoch 004/004, batch 001/002, step 0001/0496: loss=4.272143363952637\n",
"epoch 004/004, batch 001/002, step 0125/0496: loss=4.24481725692749\n",
"epoch 004/004, batch 001/002, step 0249/0496: loss=5.760097026824951\n",
"epoch 004/004, batch 001/002, step 0373/0496: loss=4.635688781738281\n"
]
}
],
"outputs": [],
"source": [
"current_lr = lr\n",
"steps = 0\n",
@ -993,7 +955,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can evaluate the _source_ simulation, and the _prediction_ of our learned hybrid solver:"
"Now we can evaluate the _source_ simulation, and the _prediction_ of our learned hybrid solver for 120 steps:"
]
},
{
@ -1020,6 +982,13 @@
"print(\"Source simulation steps \"+format(len(steps_source)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The way the evaluation is implemented above, we can only evalute a proper source simulation (i.e. one that is not modified by the NN) for `evalsteps=1`."
]
},
{
"cell_type": "code",
"execution_count": 19,
@ -1035,6 +1004,7 @@
],
"source": [
"steps_pred = [source_test]\n",
"assert(evalsteps==1) # for other settings, deactivate the source evaluation below\n",
"\n",
"for i in range(120):\n",
" my_feed_dict = { sourcesim_in: steps_pred[-1], Re_in: re_nr, lr_in: current_lr }\n",
@ -1114,7 +1084,7 @@
"source": [
"They both start out with the same initial state at $t=0$ (a downsampled solution from the reference solution manifold), and at $t=20$ the solutions still share similarities. Over time, the source version strongly diffuses the structures in the flow and looses momentum. The flow behind the obstacles becomes straight, and lacks clear vortices. \n",
"\n",
"The version produced by the hybrid solver does much better. It preserves the vortex shedding even after more than one hundred updates. Note that both outputs were produced by the same underlying solver. The second version just profits from the learned corrector which has learned to revert the overly strong dissipation of the solver. However, it also produces some visible axis-aligned structures, especially near the sides of the domain. These could be alleviated with improved training setups, e.g., more look-ahead, and a larger model.\n",
"The version produced by the hybrid solver does much better. It preserves the vortex shedding even after more than one hundred updates. Note that both outputs were produced by the same underlying solver. The second version just profits from the learned corrector which has learned to revert the overly strong dissipation of the solver. However, it also produces some visible axis-aligned structures, especially near the sides of the domain. (This could be alleviated with improved training setups, e.g., more look-ahead, and a larger model.)\n",
"\n",
"We can also compare and quantify how the models do w.r.t. reference data. The next cell plots one time step of the three versions: the reference data after 50 steps, and the re-simulated version of the source and our hybrid solver, together with a per-cell error of the two:"
]
@ -1224,9 +1194,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The overall mean absolute error (MAE) is almost 50% larger for the regular simulation. The graph below shows this behavior over time: the error of the source version is clearly above the errors of the hybrid simulator.\n",
"The overall mean absolute error (MAE) is almost 50% larger for the regular simulation. The graph below shows this behavior over time: the error of the source version is clearly larger than the errors of the hybrid simulator.\n",
"\n",
"This concludes our evaluation. Note that the improved behavior of the hybrid solver is difficult to reliably measure with simple vector norms such as an MAE or $L^2$ norm. To achieve this, we'd need to employ other, domain-specific metrics. In this case, metrics for fluids based on vorticity and turbulence properties of the flow would be applicable. However, in this text, we instead want to focus on DL-related topics and target another inverse problem with differentiable physics solvers in the next chapter."
"This concludes our evaluation. Note that the improved behavior of the hybrid solver is difficult to reliably measure with simple vector norms such as an MAE or $L^2$ norm. To improve this, we'd need to employ other, domain-specific metrics. In this case, metrics for fluids based on vorticity and turbulence properties of the flow would be applicable. However, in this text, we instead want to focus on DL-related topics and target another inverse problem with differentiable physics solvers in the next chapter."
]
},
{
@ -1268,7 +1238,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
"version": "3.8.5"
}
},
"nbformat": 4,

18
diffphys-examples.md
View File

@ -1,23 +1,23 @@
Complex Examples Overview
=======================
The following two sections with show code examples of two more complex cases that
will show what can be achieved via differentiable physics training.
The following sections will give code examples of more complex cases to
show what can be achieved via differentiable physics training.
First, we'll show a scenario that employs deep learning to learn the errors
of a numerical simulation, following Um et al. {cite}`um2020sol`.
First, we'll show a scenario that employs deep learning to represent the errors
of numerical simulations, following Um et al. {cite}`um2020sol`.
This is a very fundamental task, and requires the learned model to closely
interact with a numerical solver. Hence, it's a prime example of
situations where it's crucial to bring the numerical solver into the
deep learning loop.
Next, we'll show how to let NNs solve tough inverse problems, namely the long-term control
of a fluid simulation, following Holl et al. {cite}`holl2019pdecontrol`.
of a Navier-Stokes simulation, following Holl et al. {cite}`holl2019pdecontrol`.
This task requires long term planning,
and hence needs two networks, one to _predict_ the evolution,
and another one to _act_ to reach the desired goal.
and another one to _act_ to reach the desired goal. (Later on, in {doc}`reinflearn-code` we will compare
this approach to another DL variant using reinforcement learning.)
Both cases require quite a bit more resources than the previous examples, so you
can expect these notebooks to run longer (and it's a good idea to use the check-pointing
mechanisms when working with these examples).
can expect these notebooks to run longer (and it's a good idea to use check-pointing
when working with these examples).