李童宇,武浩然,陈衡,刘涛,李国亮
LI Tongyu,WU Haoran,CHEN Heng,LIU Tao,LI Guoliang

• 1:华北电力大学
• 2:北京国电通网络技术有限公司
• 3:国网山东省电力公司枣庄供电公司

摘要(Abstract)：

稳定配电网潮流分布、明确分布式电源的接入位置和容量是含分布式电源配电网优化运行的重要问题。提出一种基于深度强化学习算法的储能调控优化模型，实现分布式电源配置与用电负荷需求关系的匹配，从而稳定高渗透率下配电网的潮流分布。以线路损耗与电压波动性为损失函数，提出基于多目标遗传算法的分布式电源选址定容决策模型。在IEEE 14节点系统进行测试，结果表明该算法能够有效选择分布式电源的最佳接入位置和容量，在保证电压幅值不产生过大波动的同时，进一步降低了整体网络的线路损耗。
Stabilizing the power flow distribution in distribution networks and determining the connection locations and capacities of distributed generation are crucial issues in optimizing the operation of distribution networks with distributed generation. This paper proposes an energy storage scheduling and optimization model based on deep reinforcement learning(deep RL) to match the relationship between distributed energy resource allocation and electricity load demand, thereby stabilizing power flow distribution in distribution networks with high penetration rates. Using line losses and voltage fluctuations as the loss functions, the paper proposes a decision-making model for placement and sizing of distributed generation based on multi-objective genetic algorithm. Testing is conducted on the IEEE 14-bus system, and the results indicate that the algorithm can effectively select the optimal connection locations and capacities for distributed generation, reducing overall line losses while ensuring voltage amplitude remains stable.

关键词(KeyWords)： 分布式电源;深度强化学习;储能优化;多目标遗传算法;选址定容
distributed generation;deep RL;energy storage optimization;multi-objective genetic algorithm;placement and sizing

Abstract:

Keywords:

基金项目(Foundation): 国家电网有限公司科技项目（5108-202218280A-2-142-XG）

作者(Author): 李童宇,武浩然,陈衡,刘涛,李国亮
LI Tongyu,WU Haoran,CHEN Heng,LIU Tao,LI Guoliang

DOI: 10.19585/j.zjdl.202406005

参考文献(References)：

• [1]张智刚，康重庆.碳中和目标下构建新型电力系统的挑战与展望[J].中国电机工程学报，2022,42(8):2806-2819.ZHANG Zhigang,KANG Chongqing. Challenges and prospects for constructing the new-type power system towards a carbon neutrality future[J]. Proceedings of the CSEE,2022,42(8):2806-2819.
• [2]胡博，谢开贵，邵常政，等.双碳目标下新型电力系统风险评述：特征、指标及评估方法[J].电力系统自动化，2023,47(5):1-15.HU Bo,XIE Kaigui,SHAO Changzheng,et al.Commentary on risk of new power system under goals of carbon emission peak and carbon neutrality:characteristics,indices and assessment methods[J]. Automation of Electric Power Systems,2023,47(5):1-15.
• [3]康重庆，姚良忠.高比例可再生能源电力系统的关键科学问题与理论研究框架[J].电力系统自动化，2017,41(9):2-11.KANG Chongqing,YAO Liangzhong. Key scientific issues and theoretical research framework for power systems with high proportion of renewable energy[J].Automation of Electric Power Systems,2017,41(9):2-11.
• [4]陈灵，黄兴华，张功林，等.考虑削峰填谷的分布式电源集群协同控制方法[J].智慧电力，2023,51(4):8-15.CHEN Ling,HUANG Xinghua,ZHANG Gonglin,et al.Distributed generations clusters collaborative control method considering peak load shifting[J]. Smart Power,2023,51(4):8-15.
• [5]徐明宇，郝文波，王盼宝，等.基于动态随机模型的微电网群能量管理方法[J].电力工程技术，2022,41(5):140-148.XU Mingyu,HAO Wenbo,WANG Panbao,et al.Energy management method of multi-microgrids based on dynamic stochastic model[J]. Electric Power Engineering Technology,2022,41(5):140-148.
• [6]席向东，侯香臣，萨仁高娃，等.计及电压预测的主动配电网分布式电源在线优化决策方法[J].电力建设，2022,43(11):24-32.XI Xiangdong,HOU Xiangchen,SA Rengaowa,et al.Online optimization decision algorithm for distributed generations considering voltage prediction in active distribution network[J].Electric Power Construction,2022,43(11):24-32.
• [7]闫群民，穆佳豪，马永翔，等.分布式储能应用模式及优化配置综述[J].电力工程技术，2022,41(2):67-74.YAN Qunmin,MU Jiahao,MA Yongxiang,et al.Review of distributed energy storage application mode and optimal configuration[J].Electric Power Engineering Technology,2022,41(2):67-74.
• [8]刘建伟，李学斌，刘晓鸥.有源配电网中分布式电源接入与储能配置[J].发电技术，2022,43(3):476-484.LIU Jianwei,LI Xuebin,LIU Xiaoou. Distributed power access and energy storage configuration in active distribution network[J].Power Generation Technology,2022,43(3):476-484.
• [9]孟令卓超，杨锡运，赵泽宇.考虑光-荷不确定性和旋转备用约束的主动配电网经济优化调度策略[J].电力建设，2022,43(11):63-72.MENG Lingzhuochao,YANG Xiyun,ZHAO Zeyu. An economic optimal dispatch strategy for active distribution networks considering photovoltaic-load uncertainty and rotating reserve constraints[J].Electric Power Construction,2022,43(11):63-72.
• [10]邓申玮，韦钢，朱兰，等.基于区间理论含充换储一体站的主动配电网供电能力评估[J].智慧电力，2022,50(4):59-65.DENG Shenwei,WEI Gang,ZHU Lan,et al.Power supply capability evaluation of active distribution network with Charging-swapping-storage integrated station based on interval theory[J].Smart Power,2022,50(4):59-65.
• [11]李星辰，袁旭峰，李沛然，等.基于改进QPSO算法的主动配电网削峰填谷策略研究[J].电测与仪表，2022,59(2):120-125.LI Xingchen,YUAN Xufeng,LI Peiran,et al.Research on peak load shifting in active distribution network based on improved QPSD algorithm[J].Electrical Measurement&Instrumentation,2022,59(2):120-125.
• [12]谢晓帆，王可，刘秋林，等.一种考虑谐波指标的分布式电源选址定容规划方法[J].武汉大学学报（工学版），2019,52(7):616-621.XIE Xiaofan,WANG Ke,LIU Qiulin,et al. Distributed generation locating and capacity planning method considering harmonic index[J].Engineering Journal of Wuhan University,2019,52(7):616-621.
• [13] ZHANG S X,CHENG H Z,LI K,et al. Optimal siting and sizing of intermittent distributed generators in distribution system[J].IEEJ Transactions on Electrical and Electronic Engineering,2015,10(6):628-635.
• [14]王涛，刘雪飞，郑重，等.基于潮流线性化的分布式发电选址定容新算法[J].电力自动化设备，2020,40(8):117-128.WANG Tao,LIU Xuefei,ZHENG Zhong,et al.Novel locating and sizing algorithm for distributed generation based on power flow linearization[J].Electric Power Automation Equipment,2020,40(8):117-128.
• [15] JABR R A,DZAFIC I,PAL B C.Compensation in complex variables for microgrid power flow[J].IEEE Transactions on Power Systems,2018,33(3):3207-3209.
• [16]杨杉，同向前.含低电压穿越型分布式电源配电网的短路电流计算方法[J].电力系统自动化，2016,40(11):93-99.YANG Shan,TONG Xiangqian.Short-circuit current calculation of distribution network containing distributed generators with capability of low voltage ride through[J].Automation of Electric Power Systems,2016,40(11):93-99.
• [17]陈一丰，唐坤杰，董树锋，等.输配一体化潮流计算收敛性分析及提升方法[J].中国电机工程学报，2022,42(20):7524-7535.CHEN Yifeng,TANG Kunjie,DONG Shufeng,et al.Convergence analysis and promotion method of power flow calculation of integrated transmission and distribution networks[J].Proceedings of the CSEE,2022,42(20):7524-7535.
• [18] MUMTAZ F,SYED M H,AL HOSANI M,et al. A novel approach to solve power flow for islanded microgrids using modified Newton raphson with droop control of DG[J].IEEE Transactions on Sustainable Energy,2016,7(2):493-503.
• [19]杨博，王俊婷，俞磊，等.基于孔雀优化算法的配电网储能系统双层多目标优化配置[J].上海交通大学学报，2022,56(10):1294-1307.YANG Bo,WANG Junting,YU Lei,et al.Peafowl optimization algorithm based Bi-level multi-objective optimal allocation of energy storage systems in distribution network[J].Journal of Shanghai Jiao Tong University,2022,56(10):1294-1307.
• [20]刘自发，于普洋，李颉雨.计及运行特性的配电网分布式电源与广义储能规划[J].电力自动化设备，2023,43(3):72-79.LIU Zifa,YU Puyang,LI Jieyu. Planning of distributed generation and generalized energy storage in distribution network considering operation characteristics[J]. Electric Power Automation Equipment,2023,43(3):72-79.
• [21] ZANG H,CHENG L,DING T,et al.Day-ahead photovoltaic power forecasting approach based on deep convolutional neural networks and meta learning[J].International Journal of Electrical Power&Energy Systems,2020,118:105790.
• [22] HOSSAIN M A,CHAKRABORTTY R K,ELSAWAH S,et al. Predicting wind power generation using hybrid deep learning with optimization[J].IEEE Transactions on Applied Superconductivity,2021,31(8):1-5.
• [23]刘倩倩，刘钰山，温烨婷，等.基于PCC-LSTM模型的短期负荷预测方法[J].北京航空航天大学学报，2022,48(12):2529-2536.LIU Qianqian,LIU Yushan,WEN Yeting,et al. Shortterm load forecasting method based on PCC-LSTM model[J].Journal of Beijing University of Aeronautics and Astronautics,2022,48(12):2529-2536.
• [24]王欢，李鹏，曹敏，等.基于CNN-BiLSTM的长短期电力负荷预测方法[J].计算机仿真，2022,39(3):96-103.WANG Huan,LI Peng,CAO Min,et al.Long-term and short-term power load online prediction method based on CNN-BiLSTM[J]. Computer Simulation,2022,39(3):96-103.
• [25] PENG W,XU L W,LI C D,et al.Stacked autoencoders and extreme learning machine based hybrid model for electrical load prediction[J]. Journal of Intelligent&Fuzzy Systems,2019,37(4):5403-5416.
• [26] ESKANDARI H,IMANI M,MOGHADDAM M P.Convolutional and recurrent neural network based model for short-term load forecasting[J]. Electric Power Systems Research,2021,195:107173.
• [27]陈亭轩，徐潇源，严正，等.基于深度强化学习的光储充电站储能系统优化运行[J].电力自动化设备，2021,41(10):90-98.CHEN Tingxuan,XU Xiaoyuan,YAN Zheng,et al.Optimal operation based on deep reinforcement learning for energy storage system in photovoltaic-storage charging station[J].Electric Power Automation Equipment,2021,41(10):90-98.
• [28] LITTMAN M L.Markov decision processes[M]//International Encyclopedia of the Social&Behavioral Sciences.Amsterdam:Elsevier,2001:9240-9242.
• [29] NAKABI T A,TOIVANEN P.Deep reinforcement learning for energy management in a microgrid with flexible demand[J].Sustainable Energy,Grids and Networks,2021,25:100413.
• [30]段俊东，薛静杰，栗维冰.基于牛拉法的预估校正潮流计算算法[J].河南理工大学学报（自然科学版），2015,34(3):396-399.DUAN Jundong,XUE Jingjie,LI Weibing. A predictorcorrecting power flow algorithm based on NewtonRaphson method[J].Journal of Henan Polytechnic University(Natural Science),2015,34(3):396-399.