TY - JOUR

T1 - Physics-informed attention-based neural network for hyperbolic partial differential equations

T2 - application to the Buckley–Leverett problem

AU - Rodriguez-Torrado, Ruben

AU - Ruiz, Pablo

AU - Cueto-Felgueroso, Luis

AU - Green, Michael Cerny

AU - Friesen, Tyler

AU - Matringe, Sebastien

AU - Togelius, Julian

N1 - Publisher Copyright:
© 2022, The Author(s).

PY - 2022/12

Y1 - 2022/12

N2 - Physics-informed neural networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs) and are in principle capable of modeling a large variety of differential equations. PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called physics-informed attention-based neural networks (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside the convex hull of the training set.

AB - Physics-informed neural networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs) and are in principle capable of modeling a large variety of differential equations. PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called physics-informed attention-based neural networks (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside the convex hull of the training set.

UR - http://www.scopus.com/inward/record.url?scp=85129524717&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85129524717&partnerID=8YFLogxK

U2 - 10.1038/s41598-022-11058-2

DO - 10.1038/s41598-022-11058-2

M3 - Article

C2 - 35534639

AN - SCOPUS:85129524717

VL - 12

JO - Scientific Reports

JF - Scientific Reports

SN - 2045-2322

IS - 1

M1 - 7557

ER -