夏旭华,杨建迪,施永涛
XIA Xuhua,YANG Jiandi,SHI Yongtao

• 1:国网浙江杭州市富阳区供电有限公司
• 2:杭州电子科技大学自动化学院

摘要(Abstract)：

建筑综合能源系统在满足用户侧多元负荷需求的同时，能够有效提升能效比，降低建筑的碳排放量。为进一步提高建筑综合能源系统的能源调度能力，提出一种基于深度强化学习的多能流建筑综合能源系统低碳经济优化调度方法。首先，建立可充分表征能源互动耦合特性的光储一体多能流建筑综合能源系统数学模型。其次，结合深度强化学习设计建筑综合能源系统运行调度策略的状态空间、动作空间和奖励函数，运用“柔性行动器-评判器”算法搭建低碳经济优化调度框架。最后，将所提方法应用到实际夏冬季典型日负荷场景中进行验证，结果表明：与同类方法相比，所提方法收敛速度更快、优化效果更稳定，能有效降低综合能源系统日内运行的能源成本及碳排放成本。
Building integrated energy systems(BIESs) can enhance energy efficiency ratio(EER) and reduce carbon emissions while meeting diverse user-side load demands. To further improve the energy dispatch capability of BIES, this paper proposes a low-carbon economic and optimal dispatch method for BIES with multi-energy flow coupling based on deep reinforcement learning(deep RL). Firstly, a mathematical model of a photovoltaic-storage integrated BIES with multi-energy flow coupling is established to fully characterize energy interaction and coupling characteristics. Secondly, the state space, action space, and reward function for the operational dispatch strategy are designed using deep RL, and a low-carbon economic and optimal dispatch framework is constructed using the soft actor-critic(SAC) algorithm. Finally, the proposed method is validated in typical daily load scenarios in summer and winter. Results demonstrate that, compared to similar methods, the proposed method achieves faster convergence, more stable optimization effects, and effectively reduces both daily energy costs and carbon emission costs in IES operations.

关键词(KeyWords)： 深度强化学习;综合能源系统;调度优化;碳排放
deep RL;IES;scheduling optimization;carbon emission

Abstract:

Keywords:

基金项目(Foundation): 浙江省重点研发计划项目（2024C01018）

作者(Author): 夏旭华,杨建迪,施永涛
XIA Xuhua,YANG Jiandi,SHI Yongtao

DOI: 10.19585/j.zjdl.202505010

参考文献(References)：

• [1]陈姜全，李效顺，耿艺伟，等.中国城镇化与碳排放机制分析及实证研究：基于经济发展、人口转移和城镇扩张视角[J].自然资源学报，2024,39(6):1399-1417.CHEN Jiangquan,LI Xiaoshun,GENG Yiwei,et al.Mechanism analysis and empirical research on urbanization and carbon emissions in China:Based on the perspective of economic development, population transfer,and urban expansion[J].Journal of Natural Resources,2024,39(6):1399-1417.
• [2]陈勇，芮俊，肖雷鸣，等.基于动态主从博弈模型的综合能源系统碳交易方法[J].浙江电力，2024,43(4):51-62.CHEN Yong,RUI Jun,XIAO Leiming,et al. A carbon trading method for integrated energy systems based on a dynamic masterslave game model[J]. Zhejiang Electric Power,2024,43(4):51-62.
• [3]贾宏杰，雷雨，靳小龙，等.考虑绝热性能差异的建筑群与社区综合能源系统协同优化[J].电力系统自动化，2023,47(24):31-38.JIA Hongjie,LEI Yu,JIN Xiaolong,et al. Collaborative optimization of building aggregation and integrated community energy system considering difference in insulation performance[J].Automation of Electric Power Systems,2023,47(24):31-38.
• [4]任炬光，张力，金立，等.考虑可再生能源消纳的建筑综合能源系统日前经济调度模型[J].工程科学与技术，2023,55(2):160-170.REN Juguang,ZHANG Li,JIN Li,et al.Day-ahead economic dispatch model of building integrated energy systems considering the renewable energy consumption[J].Advanced Engineering Sciences,2023,55(2):160-170.
• [5] ZHU Y A,WU S Q,LI J Y,et al. Towards a carbonneutral community:integrated renewable energy systems(IRES)-sources,storage,optimization,challenges,strategies and opportunities[J]. Journal of Energy Storage,2024,83:110663.
• [6]曲婷婷.考虑冰蓄冷和冷/热负荷需求的园区综合能源系统建模与运行策略研究[D].杭州：浙江大学，2023.QU Tingting. Research on the modeling and operation strategy of integrated energy system in the park considering for lce storage and cold/heat load demand[D].Zhejiang University,2023.
• [7] ALABI T M,AGHIMIEN E I,AGBAJOR F D,et al.A review on the integrated optimization techniques and machine learning approaches for modeling, prediction,and decision making on integrated energy systems[J].Renewable Energy,2022,194:822-849.
• [8] CUCCA G,IANAKIEV A.Assessment and optimisation of energy consumption in building communities using an innovative co-simulation tool[J].Journal of Building Engineering,2020,32:101681.
• [9] PERWEZ U,YAMAGUCHI Y,MA T,et al.Multi-scale GIS-synthetic hybrid approach for the development of commercial building stock energy model[J].Applied Energy,2022,323:119536.
• [10]章禹，郭创新，尹建兵，等.区域综合能源系统电-气多元储能的优化配置研究[J].浙江电力，2023,42(6):60-69.ZHANG Yu,GUO Chuangxin,YIN Jianbing,et al. Research on the optimal configuration of an electricity-gas multi-energy storage system of the regional integrated energy system[J]. Zhejiang Electric Power,2023,42(6):60-69.
• [11]胥栋，李逸超，李赟，等.基于深度强化学习的多能流楼宇低碳调度方法[J].浙江电力，2024,43(2):126-136.XU Dong,LI Yichao,LI Yun,et al.A low-carbon scheduling method for multi-energy flow buildings based on deep reinforcement learning[J].Zhejiang Electric Power,2024,43(2):126-136.
• [12] HAN J Y,VARTOSH A.Multi-objective grasshopper optimization algorithm for optimal energy scheduling by considering heat as integrated demand response[J]. Applied Thermal Engineering,2023,234:121242.
• [13]李斌，李岩，刘佳鑫.基于改进粒子群算法的综合能源系统经济优化运行[J].电工技术，2023(2):153-157.LI Bin,LI Yan,LIU Jiaxin.Economic optimization operation of integrated energy system based on improved particle swarm optimization algorithm[J]. Electric Engineering,2023(2):153-157.
• [14] KATHIRGAMANATHAN A,MANGINA E,FINN D P.Development of a Soft Actor Critic deep reinforcement learning approach for harnessing energy flexibility in a Large Office building[J].Energy and AI,2021,5:100101.
• [15] XU L,RUAN X B,MAO C X,et al.An improved optimal sizing method for wind-solar-battery hybrid power system[J]. IEEE Transactions on Sustainable Energy,2013,4(3):774-785.
• [16] HU K Y,WANG B,CAO S H,et al.A novel model predictive control strategy for multi-time scale optimal scheduling of integrated energy system[J]. Energy Reports,2022,8:7420-7433.
• [17] LU X H,LIU Z X,MA L,et al.A robust optimization approach for optimal load dispatch of community energy hub[J].Applied Energy,2020,259:114195.
• [18]中华人民共和国住房和城乡建设部.建筑碳排放计算标准：GB/T 51366—2019[S].北京：中国建筑工业出版社，2019.
• [19] YI Z G,LUO Y S,WESTOVER T,et al.Deep reinforcement learning based optimization for a tightly coupled nuclear renewable integrated energy system[J]. Applied Energy,2022,328:120113.
• [20] YANG T,ZHAO L Y,LI W,et al.Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning[J].Energy,2021,235:121377.
• [21]张芳弛.基于状态改进-DDPG的区域综合能源系统运行优化[D].大连：大连理工大学，2021.ZHANG Fangchi.Operational optimization of regional integrated energy system based on state enhanced deep deterministic policy gradient algorithm[D]. Dalian:Dalian University of Technology,2021.
• [22] NIU D X,YU M,SUN L J,et al.Short-term multi-energy load forecasting for integrated energy systems based on CNN-BiGRU optimized by attention mechanism[J]. Applied Energy,2022,313:118801.
• [23] WANG M Y,WANG R Q,LIU J Y,et al.Operation optimization for park with integrated energy system based on integrated demand response[J].Energy Reports,2022,8:249-259.
• [24] GUO R,YE H W,ZHAO Y. Low carbon dispatch of electricity-gas-thermal-storage integrated energy system based on stepped carbon trading[J]. Energy Reports,2022,8:449-455.