李忠凡,陈曦,黄海涛
LI Zhongfan,CHEN Xi,HUANG Haitao

• 1:上海电力大学电气工程学院
• 2:国网上海物资公司

摘要(Abstract)：

随着MPIES（多园区综合能源系统）的快速发展，各园区作为独立利益主体存在利益诉求差异，且在协作过程中面临无法获取全局信息、注重隐私保护与独立自治等问题。为解决这些挑战，提出一种基于能源共享商业模式和多智能体分层强化学习的MPIES优化运行模型。首先，基于商业画布模块化设计能源共享商业模式，并构建适应MPIES多主体特性的多智能体分层强化学习协作框架，其中上层协调智能体负责进行全局资源优化和能源共享协调，而下层执行智能体负责执行能源管理并进行能源共享决策。其次，建立MPIES多智能体协作优化模型，并应用ADMM（交替方向乘子法）分布式求解，实现去中心化的全局优化。仿真结果表明，该框架下MPIES运行经济效益和资源利用效率显著提升，降低了对外部能源市场的依赖，提高了可再生能源的消纳能力。
With the rapid development of multi-park integrated energy systems(MPIESs), individual parks, as independent stakeholders, exhibit differing interests and face challenges such as limited access to global information, a strong emphasis on privacy protection, and the need for autonomy in collaborative processes. To address these issues, this paper proposes an optimal operation model for MPIESs based on an energy-sharing business model and hierarchical multi-agent reinforcement learning. First, an energy-sharing business model is designed using a business model canvas, and a collaboration framework based on hierarchical multi-agent reinforcement learning is developed to accommodate the MPIES featuring multiple agents. In this framework, the upper-layer coordinating agents optimizes global resources and coordinates energy sharing, while the lower-layer executing agents manages energy operations and makes energy-sharing decisions. Second, a multi-agent collaborative optimization model for MPIESs is developed and solved using the alternating direction method of multipliers(ADMM) to achieve decentralized global optimization. Simulation results demonstrate that the proposed framework significantly enhances the operational economic benefits and resource utilization efficiency of MPIESs, reduces dependence on external energy markets, and enhances renewable energy integration.

关键词(KeyWords)： 多园区综合能源系统;能源共享;协作优化;多智能体;分布式优化
MPIES;energy sharing;collaboration optimization;multi-agent;distributed optimization

Abstract:

Keywords:

基金项目(Foundation): 国家自然科学基金(U2066214)

作者(Author): 李忠凡,陈曦,黄海涛
LI Zhongfan,CHEN Xi,HUANG Haitao

DOI: 10.19585/j.zjdl.202509005

参考文献(References)：

• [1]王少洪.碳达峰目标下我国能源转型的现状、挑战与突破[J].价格理论与实践，2021(8):82-86.WANG Shaohong.Current situation,challenge and breakthrough of China’s energy transformation under the goal of carbon peak[J].Price(Theory&Practice),2021(8):82-86.
• [2]任喜洋，邓锋，高兵，等.推动能源资源结构向绿色低碳转型[J].中国国土资源经济，2021,34(12):48-54.REN Xiyang,DENG Feng,GAO Bing,et al.Promote the transformation of energy and resource structure to green and low-carbon development[J]. Natural Resource Economics of China,2021,34(12):48-54.
• [3]王佳蕊，孙勇，胡枭，等.基于MICP的多能耦合综合能源系统可再生能源消纳能力研究[J].电力建设，2023,44(8):157-170.WANG Jiarui,SUN Yong,HU Xiao,et al.Research on renewable energy absorption capacity of multi-energy coupling integrated energy systems based on MICP[J].Electric Power Construction,2023,44(8):157-170.
• [4]罗舒钰，李奇，阳洋，等.计及电-热-氢差异化激励需求响应的园区综合能源系统优化调度[J].电力自动化设备，2023,43(12):214-221.LUO Shuyu,LI Qi,YANG Yang,et al.Community integrated energy system optimal scheduling considering differentiated power-heat-hydrogen incentive based demand response[J]. Electric Power Automation Equipment,2023,43(12):214-221.
• [5] WANG J X,SUN Y H,MAHFOUD R J,et al.Integrated modeling of regional and park-level multi-heterogeneous energy systems[J].Energy Reports,2022,8:3141-3155.
• [6]王怡，王小君，孙庆凯，等.基于能量共享的综合能源系统群多主体实时协同优化策略[J].电力系统自动化，2022,46(4):56-65.WANG Yi,WANG Xiaojun,SUN Qingkai,et al.Multiagent real-time collaborative optimization strategy for integrated energy system group based on energy sharing[J].Automation of Electric Power Systems,2022,46(4):56-65.
• [7]黎静华，朱梦姝，陆悦江，等.综合能源系统优化调度综述[J].电网技术，2021,45(6):2256-2272.LI Jinghua,ZHU Mengshu,LU Yuejiang,et al.Review on optimal scheduling of integrated energy systems[J].Power System Technology,2021,45(6):2256-2272.
• [8]陈燚，何山.多园区综合能源系统互联两阶段分布式运行优化策略[J].电网与清洁能源，2024,40(8):18-25.CHEN Yi,HE Shan. Optimization strategy of the twostage distributed operation for multi-park integrated energy system interconnection[J].Power System and Clean Energy,2024,40(8):18-25.
• [9]丁世飞，杜威，张健，等.多智能体深度强化学习研究进展[J].计算机学报，2024,47(7):1547-1567.DING Shifei,DU Wei,ZHANG Jian,et al.Research progress of multi-agent deep reinforcement learning[J]. Chinese Journal of Computers,2024,47(7):1547-1567.
• [10] YANG Y D,WANG J.An overview of multi-agent reinforcement learning from game theoretical perspective[EB/OL]. 2020:2011.00583. https：//arxiv. org/abs/2011.00583v3.
• [11]孙庆凯，王小君，王怡，等.基于多智能体Nash-Q强化学习的综合能源市场交易优化决策[J].电力系统自动化，2021,45(16):124-133.SUN Qingkai,WANG Xiaojun,WANG Yi,et al.Optimal trading decision-making for integrated energy market based on multi-agent Nash-Q reinforcement learning[J].Automation of Electric Power Systems,2021,45(16):124-133.
• [12]张帆，武东昊，陈玉萍，等.多智能体深度强化学习的分布式园区综合能源系统经济调度策略[J].电力系统及其自动化学报，2022,34(12):18-26.ZHANG Fan,WU Donghao,CHEN Yuping,et al. Economic scheduling strategy for integrated energy system in distributed parks based on multi-agent deep reinforcement learning[J]. Proceedings of the CSU-EPSA,2022,34(12):18-26.
• [13]董雷，刘雨，乔骥，等.基于多智能体深度强化学习的电热联合系统优化运行[J].电网技术，2021,45(12):4729-4738.DONG Lei,LIU Yu,QIAO Ji,et al.Optimal dispatch of combined heat and power system based on multi-agent deep reinforcement learning[J]. Power System Technology,2021,45(12):4729-4738.
• [14]刘俊峰，王晓生，卢俊菠，等.基于多主体博弈和强化学习的多微网系统协同优化研究[J].电网技术，2022,46(7):2722-2732.LIU Junfeng,WANG Xiaosheng,LU Junbo,et al.Collaborative optimization of multi-microgrid system based on multi-agent game and reinforcement learning[J]. Power System Technology,2022,46(7):2722-2732.
• [15]叶宇剑，袁泉，刘文雯，等.基于参数共享机制多智能体深度强化学习的社区能量管理协同优化[J].中国电机工程学报，2022,42(21):7682-7695.YE Yujian,YUAN Quan,LIU Wenwen,et al.Parameter sharing empowered multi-agent deep reinforcement learning for coordinated management of energy communities[J].Proceedings of the CSEE,2022,42(21):7682-7695.
• [16]（瑞士）亚历山大·奥斯特瓦德（Alexander Osterwalder），（比）伊夫·皮尼厄（Yves Pigneur）.商业模式新生代[M].北京：机械工业出版社，2011.
• [17]张磊，吴红斌，何叶，等.基于深度强化学习的氢能综合能源系统优化调度方法[J].电力系统自动化，2024,48(16):132-141.ZHANG Lei,WU Hongbin,HE Ye,et al.Optimal scheduling method for integrated energy systems with hydrogen based on deep reinforcement learning[J]. Automation of Electric Power Systems,2024,48(16):132-141.
• [18] KULKARNI T D,NARASIMHAN K R,SAEEDI A,et al. Hierarchical deep reinforcement learning:integrating temporal abstraction and intrinsic motivation[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. December 5-10,2016,Barcelona,Spain.ACM,2016:3682-3690.
• [19] XU Jing,ZHONG Fangwei,WANG Yizhou. Learning multi-agent coordinationfor enhancing target coverage in directional sensor networks[C]//Advances in Neural Information Processing Systems,2020,33:10053.