Reference
F. Ruelens,
B. J. Claessens, S. Quaiyum,
B. De Schutter, R. Babuška, and R. Belmans, "Reinforcement learning
applied to an electric water heater: From theory to practice,"
IEEE Transactions on Smart Grid, vol. 9, no. 4, pp.
3792-3800, 2018.
Abstract
Electric water heaters have the ability to store energy in their water buffer
without impacting the comfort of the end user. This feature makes them a prime
candidate for residential demand response. However, the stochastic and
non-linear dynamics of electric water heaters, makes it challenging to harness
their flexibility. Driven by this challenge, this paper formulates the
underlying sequential decision-making problem as a Markov decision process and
uses techniques from reinforcement learning. Specifically, we apply an
auto-encoder network to find a compact feature representation of the sensor
measurements, which helps to mitigate the curse of dimensionality. A well-known
batch reinforcement learning technique, fitted Q-iteration, is used to find a
control policy, given this feature representation. In a simulation-based
experiment using an electric water heater with 50 temperature sensors, the
proposed method was able to achieve good policies much faster than when using
the full state information. In a lab experiment, we apply fitted Q-iteration to
an electric water heater with eight temperature sensors. Further reducing the
state vector did not improve the results of fitted Q-iteration. The results of
the lab experiment, spanning 40 days, indicate that compared to a thermostat
controller, the presented approach was able to reduce the total cost of energy
consumption of the electric water heater by 15%.
Publisher page
Downloads
BibTeX
@article{RueCla:16-028,
author = {Ruelens, Frederik and Claessens, Bert J. and Quaiyum, Salman and
De Schutter, Bart and Babu{\v{s}}ka, Robert and Belmans, Ronnie},
title = {Reinforcement Learning Applied to an Electric Water Heater:
{From} Theory to Practice},
journal = {IEEE Transactions on Smart Grid},
volume = {9},
number = {4},
pages = {3792--3800},
year = {2018}
}