TY - GEN
T1 - State Constrained Stochastic Optimal Control Using LSTMs
AU - Dai, Bolun
AU - Krishnamurthy, Prashanth
AU - Papanicolaou, Andrew
AU - Khorrami, Farshad
N1 - Publisher Copyright:
© 2021 American Automatic Control Council.
PY - 2021/5/25
Y1 - 2021/5/25
N2 - In this paper, we propose a new methodology for state constrained stochastic optimal control (SOC) problems. The solution is based on past work in solving SOC problems using forward-backward stochastic differential equations (FB-SDE). Our approach in solving the FBSDE utilizes a deep neural network (DNN), specifically Long Short-Term Memory (LSTM) networks. LSTMs are chosen to solve the FBSDE to address the curse of dimensionality, non-linearities, and long time horizons. In addition, the state constraints are incorporated using a hard penalty function, resulting in a controller that respects the constraint boundaries. Numerical instability that would be introduced by the penalty function is dealt with through an adaptive update scheme. The control design methodology is applicable to a large class of control problems. The performance and scalability of our proposed algorithm are demonstrated by numerical simulations.
AB - In this paper, we propose a new methodology for state constrained stochastic optimal control (SOC) problems. The solution is based on past work in solving SOC problems using forward-backward stochastic differential equations (FB-SDE). Our approach in solving the FBSDE utilizes a deep neural network (DNN), specifically Long Short-Term Memory (LSTM) networks. LSTMs are chosen to solve the FBSDE to address the curse of dimensionality, non-linearities, and long time horizons. In addition, the state constraints are incorporated using a hard penalty function, resulting in a controller that respects the constraint boundaries. Numerical instability that would be introduced by the penalty function is dealt with through an adaptive update scheme. The control design methodology is applicable to a large class of control problems. The performance and scalability of our proposed algorithm are demonstrated by numerical simulations.
UR - http://www.scopus.com/inward/record.url?scp=85111215041&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85111215041&partnerID=8YFLogxK
U2 - 10.23919/ACC50511.2021.9482832
DO - 10.23919/ACC50511.2021.9482832
M3 - Conference contribution
AN - SCOPUS:85111215041
T3 - Proceedings of the American Control Conference
SP - 1294
EP - 1299
BT - 2021 American Control Conference, ACC 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 American Control Conference, ACC 2021
Y2 - 25 May 2021 through 28 May 2021
ER -