Discussing the article: "Neural networks made easy (Part 82): Ordinary Differential Equation models (NeuralODE)"

[MetaQuotes]

Check out the new article: Neural networks made easy (Part 82): Ordinary Differential Equation models (NeuralODE).

In this article, we will discuss another type of models that are aimed at studying the dynamics of the environmental state.

Let's get acquainted with a new model family: Ordinary Differential Equations. Instead of specifying a discrete sequence of hidden layers, they parameterize the derivative of the hidden state using a neural network. The results of the model are calculated using a "black box", that is, the Differential Equation Solver. These continuous-depth models use a constant amount of memory and adapt their estimation strategy to each input signal. Such models were first introduced in the paper "Neural Ordinary Differential Equations". In this paper, the authors of the method demonstrate the ability to scale backpropagation using any Ordinary Differential Equation (ODE) solver without access to its internal operations. This enables end-to-end training of ODEs within larger models.

Author: Dmitriy Gizlyk