胥栋,李逸超,李赟,徐刚,杜佳玮
XU Dong,LI Yichao,LI Yun,XU Gang,DU Jiawei

• 1:国网上海市电力公司浦东供电公司

摘要(Abstract)：

建筑减排已成为中国达到“双碳”目标的重要途径，智慧楼宇作为多能流网络耦合的综合能源主体，面临碳排放量较多、多能流网络耦合程度高、负荷用能行为动态特性明显等问题。针对这一问题，提出基于深度强化学习的多能流楼宇低碳调度方法。首先，根据智慧楼宇的实际碳排放量，建立了一种奖惩阶梯型碳排放权交易机制。其次，面向碳市场和多能流耦合网络，以最小化运行成本为目标函数，建立多能流低碳楼宇调度模型，并将该调度问题转换为马尔可夫决策过程。然后，利用Rainbow算法进行优化调度问题的求解。最后，通过仿真分析验证了优化调度模型的可行性及有效性。
Building emissions reduction has become a crucial pathway for China to achieve its 'dual-carbon' goals.As an integrated energy entity coupled with multi-energy flow networks, smart buildings face challenges such as high carbon emissions, a high degree of coupling in multi-energy flow networks, and distinct dynamic characteristics in load energy consumption behavior. In response to these challenges, a low-carbon scheduling method for multienergy flow buildings based on deep reinforcement learning(deep RL) is proposed. Firstly, a reward and punishment ladder-type carbon emissions trading mechanism is established based on the actual carbon emissions of smart buildings. Secondly, targeting the carbon market and multi-energy flow coupling networks, a low-carbon scheduling model for multi-energy flow buildings is developed, aiming to minimize operating costs as the objective function, and the scheduling is transformed into a Markov decision process(MDP). Subsequently, the Rainbow algorithm is employed to solve the optimal scheduling. Finally, the feasibility and effectiveness of the optimal scheduling model are verified through simulation analysis.

关键词(KeyWords)： “双碳”目标;多能流;低碳调度;深度强化学习
'dual-carbon' goals;multi-energy flow;low-carbon scheduling;deep RL

Abstract:

Keywords:

基金项目(Foundation): 国网上海市电力公司浦东供电公司营销项目（640921220001）

作者(Author): 胥栋,李逸超,李赟,徐刚,杜佳玮
XU Dong,LI Yichao,LI Yun,XU Gang,DU Jiawei

DOI: 10.19585/j.zjdl.202402014

参考文献(References)：

• [1]周孝信，陈树勇，鲁宗相，等.能源转型中我国新一代电力系统的技术特征[J].中国电机工程学报，2018,38(7):1893-1904.ZHOU Xiaoxin,CHEN Shuyong,LU Zongxiang,et al.Technology features of the new generation power system in China[J].Proceedings of the CSEE,2018,38(7):1893-1904.
• [2]卢治霖，刘明波，尚楠，等.考虑碳排放权交易市场影响的日前电力市场两阶段出清模型[J].电力系统自动化，2022,46(10):159-170.LU Zhilin,LIU Mingbo,SHANG Nan,et al. Two-stage clearing model for day-ahead electricity market considering impact of carbon emissions trading market[J].Automation of Electric Power Systems,2022,46(10):159-170.
• [3]曹丽霞.计及多元储能的楼宇综合能源系统优化调度[D].北京：华北电力大学，2022.CAO Lixia.Optimal scheduling of building integrated energy system considering multi-energy storage[D].Beijing:North China Electric Power University,2022.
• [4]张雪纯，高广玲，张智晟，等.基于需求响应的建筑楼宇综合能源系统优化调度[J].电力需求侧管理，2019,21(4):28-34.ZHANG Xuechun,GAO Guangling,ZHANG Zhisheng,et al.Optimal scheduling of building integrated energy system based on demand response[J]. Power Demand Side Management,2019,21(4):28-34.
• [5]张宇桐.考虑楼宇不同供热模式的区域电-热综合能源系统分布式协调优化调度[D].吉林：东北电力大学，2022.ZHANG Yutong.Distributed coordinated optimal scheduling of district electric-heat integrated energy system considering different heating modes of buildings[D]. Jilin:Northeast Dianli University,2022.
• [6]余苏敏，杜洋，史一炜，等.考虑V2B智慧充电桩群的低碳楼宇优化调度[J].电力自动化设备，2021,41(9):95-101.YU Sumin,DU Yang,SHI Yiwei,et al.Optimal scheduling of low-carbon building considering V2B smart charging pile groups[J].Electric Power Automation Equipment,2021,41(9):95-101.
• [7]范宏，于伟南，柳璐，等.双碳目标下考虑电氢互补的智慧园区多楼宇协调调度[J].电力系统自动化，2022,46(21):42-51.FAN Hong,YU Weinan,LIU Lu,et al.Multi-building coordinated dispatch in smart park for carbon emission peak and carbon neutrality considering electricity and hydrogen complementary[J]. Automation of Electric Power Systems,2022,46(21):42-51.
• [8]孙国强，陈晓东，周亦洲，等.基于分布鲁棒机会约束的多类型楼宇电-碳联合分布式交易方法[J].电网技术，2023,47(8):3078-3089.SUN Guoqiang,CHEN Xiaodong,ZHOU Yizhou,et al.Electricity-carbon joint distributed trading for multi-type buildings based on distributional robust chance constraint[J].Power System Technology,2023,47(8):3078-3089.
• [9]袁桂丽，贾新潮，陈少梁，等.虚拟电厂源-荷协调多目标优化调度[J].太阳能学报，2021,42(5):105-112.YUAN Guili,JIA Xinchao,CHEN Shaoliang,et al.Multiobjective optimal dispatch considering source-load coordination for virtual power plant[J].Acta Energiae Solaris Sinica,2021,42(5):105-112.
• [10]CUI S C,WANG Y W,SHI Y,et al.A new and fair peerto-peer energy sharing framework for energy buildings[J].IEEE Transactions on Smart Grid,2020,11(5):3817-3826.
• [11]杨昭，艾欣.考虑电能共享的综合能源楼宇群分布式优化调度[J].电网技术，2020,44(10):3769-3778.YANG Zhao,AI Xin. Distributed optimal scheduling for integrated energy building clusters considering energy sharing[J].Power System Technology,2020,44(10):3769-3778.
• [12]黄炳强.强化学习方法及其应用研究[D].上海：上海交通大学，2007.HUANG Bingqiang.Research on the reinforcement learning method and its application[D]. Shanghai:Shanghai Jiao Tong University,2007.
• [13]王统.基于强化学习的智能楼宇群能量优化方法[D].北京：华北电力大学，2021.WANG Tong. A reinforcement learning-based approach for energy optimization of smart building clusters[D].Beijing:North China Electric Power University,2021.
• [14]于一潇，杨佳峻，杨明，等.基于深度强化学习的风电场储能系统预测决策一体化调度[J].电力系统自动化，2021,45(1):132-140.YU Yixiao,YANG Jiajun,YANG Ming,et al.Prediction and decision integrated scheduling of energy storage system in wind farm based on deep reinforcement learning[J].Automation of Electric Power Systems,2021,45(1):132-140.
• [15]黄刚.中国转型升级背景下的碳配额分配机制构建思路研究[J].世界环境，2019(2):50-52.HUANG Gang. Research on the construction of carbon quota allocation mechanism under the background of China’s transformation and upgrading[J].World Environment,2019(2):50-52.
• [16]田晓丽，侯忠男.浅议我国碳排放权初始配额分配机制[J].时代金融，2022(6):76-78.TIAN Xiaoli,HOU Zhongnan.On the initial quota allocation mechanism of carbon emission rights in China[J].Times Finance,2022(6):76-78.
• [17]张潇，邵鑫潇，蒋惠琴.行业碳排放权的初始配额分配：文献综述[J].资源开发与市场，2018,34(11):1520-1525.ZHANG Xiao,SHAO Xinxiao,JIANG Huiqin.Initial allocation of carbon emission rights in the industry—literature review[J]. Resource Development&Market,2018,34(11):1520-1525.
• [18]GU Y R,HUANG X L. A reactive power optimization partially observable Markov decision process with data uncertainty using multi-agent actor-attention-critic algorithm[J]. International Journal of Electrical Power&Energy Systems,2023,147:108848.
• [19]陈柏翰，冯伟，孙凯，等.冷热电联供系统多元储能及孤岛运行优化调度方法[J].电工技术学报，2019,34(15):3231-3243.CHEN Pohan,FENG Wei,SUN Kai,et al.Multi-energy storage system and islanded optimal dispatch method of CCHP[J]. Transactions Of China Electrotechnical Society,2019,34(15):3231-3243.
• [20]邱彬，宋绍鑫，王凯，等.计及需求响应和阶梯型碳交易机制的区域综合能源系统优化运行[J].电力系统及其自动化学报，2022,34(5):87-95.QIU Bin,SONG Shaoxin,WANG Kai,et al.Optimal operation of regional integrated energy system considering demand response and ladder-type carbon trading mechanism[J]. Proceedings of the CSU-EPSA,2022,34(5):87-95.