Learning Safety in Model-Based Reinforcement Learning using MPC and Gaussian Processes

Reference

F. Airaldi, B. De Schutter, and A. Dabiri, "Learning safety in model-based reinforcement learning using MPC and Gaussian processes," Proceedings of the 22nd IFAC World Congress, Yokohama, Japan, pp. 5759-5764, July 2023.

Abstract

This paper proposes a method to encourage safety in Model Predictive Control (MPC)-based Reinforcement Learning (RL) via Gaussian Process (GP) regression. The framework consists of 1) a parametric MPC scheme that is employed as model-based controller with approximate knowledge on the real system's dynamics, 2) an episodic RL algorithm tasked with adjusting the MPC parametrization in order to increase its performance, and 3) GP regressors used to estimate, directly from data, constraints on the MPC parameters capable of predicting, up to some probability, whether the parametrization is likely to yield a safe or unsafe policy. These constraints are then enforced onto the RL updates in an effort to enhance the learning method with a probabilistic safety mechanism. Compared to other recent publications combining safe RL with MPC, our method does not require further assumptions on, e.g., the prediction model in order to retain computational tractability. We illustrate the results of our method in a numerical example on the control of a quadrotor drone in a safety-critical environment.

Publisher page

• Access full text  

Downloads

• PDF of corresponding technical report (419 KB)

BibTeX

@inproceedings{AirDeS:23-031,
   author    = {Airaldi, Filippo and De Schutter, Bart and Dabiri, Azita},
   title     = {Learning Safety in Model-Based Reinforcement Learning using
                {MPC} and {Gaussian} Processes},
   booktitle = {Proceedings of the 22nd IFAC World Congress},
   address   = {Yokohama, Japan},
   pages     = {5759--5764},
   month     = jul,
   year      = {2023}
   }

• Download BibTeX entry