叶琳,项中明,张静,徐建平,吕勤,尚秀敏,杨靖萍,刁瑞盛
YE Lin,XIANG Zhongming,ZHANG Jing,XU Jianping,LYU Qin,SHANG Xiumin,YANG Jingping,DIAO Ruisheng

• 1:国网浙江省电力有限公司
• 2:国网浙江省电力有限公司金华供电公司
• 3:国电南瑞南京控制系统有限公司

摘要(Abstract)：

随着新型电力系统规划与调控中的复杂性、动态性和不确定性持续增大，制定满足多种安全和经济约束的电网运行方式面临诸多挑战。该过程通常需要大量的人工调整和仿真计算，在高维动作空间中搜索满足电网在基态和故障工况下安全和经济要求的可行解。为此，提出一种基于多强化学习智能体架构的方法，将该问题描述为马尔可夫决策过程，通过训练集中式和分布式的强化学习智能体，自动调整不同类型的电网可控资源，从而控制电网传输线路功率，满足多种电网运行安全指标。该方法的有效性在某实际电网模型中得到了验证。
As the planning and dispatching of new types of power systems become increasingly complicated,dynamic and dubious,it is very challenging to lay down operating mode for power grid that satisfies both security and economic constraints. This process typically requires intensive manual tuning and simulation calculations to search for feasible solutions in a high-dimensional action space that fulfills the requirements on safety and economy of the grid in ground state and fault conditions. Therefore,an approach based on multi-DRL-agent architecture is proposed to describe the problem as a Markov decision process(MDP). The approach automatically adjusts different types of controllable grid resources by training centralized and distributed multi-DRL-agent to control the power of transmission lines,thus complying with various safety indicators of grid operation. The method is proved to be effective in a power grid model.

关键词(KeyWords)： 人工智能;电网调度与控制;深度强化学习;多智能体
artificial intelligence;grid dispatch and control;deep reinforcement learning;multi-agent

Abstract:

Keywords:

基金项目(Foundation): 国网浙江省电力有限公司科技项目（5211JH1900M4）

作者(Author): 叶琳,项中明,张静,徐建平,吕勤,尚秀敏,杨靖萍,刁瑞盛
YE Lin,XIANG Zhongming,ZHANG Jing,XU Jianping,LYU Qin,SHANG Xiumin,YANG Jingping,DIAO Ruisheng

DOI: 10.19585/j.zjdl.202206001

参考文献(References)：

• [1]MOHAGHEGHI E,ALRAMLAWI M,GABASH A,et al.A survey of real-time optimal power flow[J].Energies,2018,11(11):3142.
• [2]MOLZAHN D,HOLZER J,LESIEUTRE B,et al.Implementation of a large-scale optimal power flow solver based on semidefinite programming[J].IEEE Transactions on Power Systems,2013,28(4):3987-3998.
• [3]MADANI R,ASHRAPHIJUO M,LAVAEI J.Promises of conic relaxation for contingency-constrained optimal power flow problem[J].IEEE Transactions on Power Systems,2015,31(2):199-211.
• [4]TANG Y,DVIJOTHAM K,LOW S.Real time optimal power flow[J].IEEE Transactions on Smart Grid,2017,8(6):2963-2973.
• [5]DOE ARPA-E.Grid Optimization Competition[R/OL].https：//gocompetition.energy.gov/.
• [6]DIAO,R,WANG Z,SHI D,et al.Autonomous voltage control for grid operation using deep reinforcement learning[C]//Proceedings of IEEE PES General Meeting,2019.Atlanta,GA,USA:IEEE,2019:1-5.
• [7]ZHANG B,LU X,DIAO R,et al.Real-time Autonomous line flow control using proximal policy optimization[C]//Proceedings of the IEEE PES General Meeting,2020.Montreal,Canada:IEEE,2020:1-5.
• [8]YAN Z,XU Y.Data-driven load frequency control for stochastic power systems:A deep reinforcement learning method with continuous action search[J].IEEE Transactions on Power Systems,2019,34(2):1653-1656.
• [9]HUANG Q,HUANG R,HAO W,et al.Adaptive power system emergency control using deep reinforcement learning[J].IEEE Transactions on Smart Grid,2019,11(2):1171-1182.
• [10]HAARNOJA T,ZHOU A,ABBEE P,et al.Soft actorcritic:off policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proceedings of ICML,2018.Stockholm,Sweden:IMLS,2018:1801.01290.
• [11]KUNDUR P.Power system stability and control[M].New York:Mc Graw-Hill,1994.
• [12]DIAO R,SHI D,ZHANG B,et al.On training effective reinforcement learning agents for real-time power grid operation and control[C]//Proceedings of Neur IPS,2012.Online:IMLS,2012.06458.
• [13]WANG S,DIAO R,LAN T,et al.A DRL-aided multilayer stability model calibration platform considering multiple events[C]//Proceedings of the 2020 IEEE PESGeneral Meeting.Online:IEEE,2020:1-5.
• [14]LAN T,DUAN J,ZHANG B,et al.AI-Based Autonomous topology control for maximizing time-series available transfer capabilities considering uncertainties[C]//Proceedings of the 2020 IEEE PES General Meeting.Online:IEEE,2020:1-5.
• [15]WANG S,DIAO R,XU C.On multi-event co-calibration of dynamic model parameters using soft actor-critic[J].IEEE Transactions on Power Systems,2021,36(1):521-524.
• [16]王威，李润秋，张鹭，等.计及多类型电储能的综合能源系统优化运行对比分析研究[J].电网与清洁能源，2020,36(2):110-116.