Speaker
Description
Determination of future states based on the actual one is a key task in many areas of dynamic system analysis like forecasting, stability analysis or model predictive control. For this purpose, classically a mathematical model needs to be set up from physical principles. The resulting equations of motion are then usually transformed into a system of 1st-order ordinary differential equations (ODE) [1] and solved with established numerical integration methods to determine future system states starting from an initial condition.
However, as mentioned, this classical approach requires the equations of motion to be formulated, which can be rather challenging due to the required expert knowledge or even impossible if the underlying physics is unknown. In such cases, machine learning approaches may be used to first extract the system behavior from data and then train surrogates predicting future states. To make use of established numerical integration methods in the forecasting process, the surrogate should substitute the state function of the underlying ODE. This goal can be reached, e.g., by using divided differences to first generate approximations of the state derivatives and then train an ODE-surrogate as proposed in [2]. However, divided differences will fail in case of noise or for large step sizes.
To overcome these difficulties, here a new approach is proposed, which may be called Artificial Neural Ordinary Differential Equation (ANODE). An explicit numerical integration scheme is wrapped around a feed-forward neural network representing the state function of the ODE to be learned. For training the ANODE, first multiple initial conditions are forward-propagated by solving the corresponding initial value problems up to each specified termination times using any explicit numerical integration scheme. Second, losses are generated by comparing the predicted output states with the respective exact ones. Finally, these losses are used to update the weights of the embedded neural network using backpropagation methods.
The feasibility of ANODE is demonstrated for various planar systems using a fourth-order Runge-Kutta integration scheme to forecast system states based on the trained surrogate. Generally speaking, the proposed ANODE approach works with any explicit single- or multistep integration scheme, it only has to be connected to the backpropagation graph to calculate all required derivatives.
[1] Bestle, D. (1994). Analyse und Optimierung von Mehrkörpersystemen: Grundlagen und rechnergestützte Methoden. Springer, Berlin.
[2] Bestle, D. & Bielitz, T. (2024). Real‐time models for systems with costly or unknown dynamics. Proceedings in Applied Mathematics & Mechanics. https://doi.org/10.1002/pamm.202400008 .
