Training physics-informed neural networks: One learning to rule them all?

Physics-informed neural networks (PINNs) are gaining popularity as powerful tools for solving nonlinear Partial Differential Equations (PDEs) through Deep Learning. PINNs are trained by incorporating physical laws as soft constraints in the loss function. Such an approach is effective for trivial equations, but fails in solving various classes of more complex dynamical systems. In this work, we put on the test bench three state-of-the-art PINN training methods for solving three popular Partial Differential Equations (PDEs) of increasing complexity, besides the additional application of the Fourier Feature Embedding (FFE), and the introduction of a novel implementation of Curriculum regularization. Experiments evaluate the convergence of the trained PINN and its prediction error rate for different training sizes and training lengths (i.e., number of epochs). To provide an overview of the behavior of each learning method, we introduce a new metric, named overall score. Our experiments show that a given approach can either be the best in all situations or not converge at all. The same PDE can be solved with different learning methods, which in turn give the best results, depending on the training size or the use of FFE. From our experiments we conclude that there is no learning method to train them all, yet we extract useful patterns that can drive future works in this growing area of research. All code and data of this manuscript are publicly available on GitHub.