A Q-Learning Algorithm for Optimizing On-Load Tap Changer Operation and Voltage Control in Distribution Networks with High Integration of Renewable Energy Sources

Distribution networks have been experiencing significant changes under the pressure of the energy transition. The high integration of renewable energy sources combined with the electrification process introduces new challenges in managing distribution networks. Innovative solutions aimed at optimizing the control of complex problems, starting from historical data instead of a detailed system model, have been growing due to rapid development in artificial intelligence and machine learning. This paper proposes a Q-learning algorithm to control the tap setting of the on-load tap changer installed in primary substation transformers. The ultimate goal is to maintain voltage magnitudes at all busses of the medium-voltage distribution network within a safe range, simultaneously optimizing on-load tap changer operations. As a case study, the effectiveness of the proposed algorithm is assessed using a real medium-voltage distribution network with high penetration of renewable energy sources that supplies more than 2500 users/prosumers. The ability of the proposed algorithm to control bus voltages is tested in several scenarios characterized by significant variability and uncertainty. Outcomes show that the proposed algorithm is suitable for optimizing voltage control in distribution networks using a data-driven approach.