A Non Cooperative Game Theoretic Approach for Energy Management in MV grid F.Mangiatordia,b, E.Pallottia,b, P. Del Vecchiob a
Fondazione Ugo Bordoni, bUniversity of Roma TRE Rome, Italy
[email protected],
[email protected],
[email protected] Abstract— Demand-side management (DSM) is one of the key functionality of the future power grid as it enables the user to control the energy consumption for an efficient and sustainable allocation of the energy resources. In addition, the DSM promotes the integration of the new renewable sources power generation systems in the traditional electrical grid, improving the balance between local supply and energy demand. This paper proposes a novel DSM technique based on a non-cooperative game framework, to reduce the Peak to Average Ratio (PAR) of the power system, minimizing daily electricity payment of each consumer in the geographical area. Each consumer is considered like a player in an energy game and he/she is encouraged to reschedule the energy consumption, applying an MPSO algorithm to shift in time those loads occurring during peak consumption periods. The dynamic pricing policy applied by the energy providers, leads each player to adopt the best strategy among its Pareto scheduling solutions, to minimize the energy peak in the overall load demand of a geographical area. Simulation results confirm the effectiveness of this distributed game theoretical approach to the DSM problem. An appreciable PAR reduction is achieved at the price of a low information exchange between the energy provider and each consumer, keeping the user privacy safe and minimizing the overhead of signaling information over the network. Keywords— Demand Side Management, load scheduling, non cooperative game, MPSO algorithm; dynamic energy pricing, smart grid
I.
INTRODUCTION
The smart grids are the evolution of the traditional power grid in a complex cyber-physical system, which optimizes the asset utilization and the operating efficiency to build a sustainable economy for future generations by means of new products and services [1]. To accommodate these goals the future power grid needs: • new generation renewable resources,
systems
including
distributed
• advanced sensing technologies and communication networks to monitor system integrity and to exchange bidirectional flow of data between end-users and utility company, • intelligent distributed control system and new software algorithms to manage the energy consumptions at the customer side.
The future power grid has to be designed to be more dynamic in its configuration and in its operational condition [2] to dispatch fast-ramping energy generation from renewable source and balance variations in local load demand. DemandSide Management (DMS) techniques and distributed energy storage devices are key facilitators for the deployment of a dynamic smart power infrastructure [3-5]. Demand side management comprises different programs implemented by utilities to influence energy consumption patterns, promote efficient use of available energy, avoid long-distance transport of locally generated energy and maintain the reliability of the grid without installing new generation and transmission capacity. Various kind of DSM methods are found in the literature, with different applied measures, control process and energy management algorithms [6]. More recent distributed DSM strategies are taking an increasing relevance. They suggest DSM frameworks in which the utilities do not remotely control the energy consumption of certain household appliances but involve the end-users in the direct management of their loads to improve the billings and the peaks of the aggregated electricity demand in a geographical area. In this context, game theory and multi-agent systems can be used to model the interactions in the distributed decision making process on the management of energy resources [7-9]. This paper addresses the development of a distributed framework for demand-side management using the noncooperative game theoretical approach. Multi-agent systems are embedded in user’s smart meters and represent the players of the DSM game. They get information on the dynamic price of energy from the utility and consequently they select the best strategy minimizing their energy cost, shaving the Peak to Average Ratio (PAR) of the power grids in that area. For each customer the set of possible strategies are constituted by a finite set of the Pareto solutions in a Multi-Objective PSO scheduling problem of the daily energy consumptions. The proposed DSM system lightens the signaling burden in the grid and protects user’s privacy. The organization of the paper is as follows: Section II explains the proposed DSM framework model; Section III describes the MPSO planning of user consumptions and the algorithm to solve the non cooperative energy game; Section IV presents and analyzes the simulation results.; Section V concludes the paper.
978-1-4799-2803-3/13/$31.00 ©2013 IEEE
II.
,…
DSM FRAMEWORK MODEL
This paper consider the MV grid of a small geographical area that conveys energy from one power plant to a set of N end-users, each controlled by a Multi-Agent System (MAS). The end-user can be equipped with battery storing energy from renewable sources (solar panels or micro-wind turbines). The distributed DSM framework can be viewed as a collection of MAS, integrated in smart meter, that have computational intelligence capabilities and two-way communication technologies. Each MAS is connected with a user interface to collect the tasks planned for the current day and gets information about power-consumption-patterns of individual appliances from the smart meter. Besides a MAS gains knowledge of timedifferentiated-price of energy from the utility company to schedule its power demand of daily tasks applying the MPSO algorithm and takes part in a non-cooperative game to reduce the Peak-to-Average Ratio in the geographical area when the Nash equilibrium for the game is achieved. The PAR is a relevant parameter for the reliability of the power grid, and the balancing between energy demand and supply, during each hour of the day inside the considered geographical area. A high value of PAR may require more intensive generation capacity. The following subsection describes the mathematical formulation of the non-cooperative game involving the MAS. A. Mathematical Game Formulation In DSM decentralized architecture, each MAS is a player of the non-cooperative game and it is associated with a end consumer of a power grid. Let denote the set of actions of the player. The … vector of actions , with … , is called an action profile and is referred also a strategy profile. Hence, the DSM game can be defined in strategic form as a tuple ,
,
(1)
where is the set of player with | | , is the strategy space of player , is the utility function also named payoff function of the player . , The utility function depends on the action by player and on the specified action profile users.
(2)
(3)
,…
denotes the total load of user for each time slot The value ∆ and is obtained as : ∑
(4) (5)
where is the vector of time-slot-consumption of the . scheduled task The Multi-Agent System obtains the vectors for the set after finding the Pareto solutions of the of daily tasks MPSO scheduling problem. Hence the MAS is able to specifies the set of actions . Given N end-users of power grid and one action profile … , the total load for each time slot is commonly expressed as [9]: ∑
(6)
and the daily Peak-to-Average Ratio is calculated as: PAR =
max t∈T Lt
∑
T
L t=1 t
(7)
T
If it is defined the average daily load of user as: (8) then PAR can be also expressed as: PAR =
max t∈T Lt
∑
N i=1
Pi
(9)
The denominator of (9) is constant since depends only from the set of daily tasks and not from the particular load profile of the user. Basing on the previous definition, the utility function , for each player can be expressed as:
selected of the other (10)
To define the utility function, some preliminary definitions are provided. So, for the user , the set of actions consist of load profiles derived by running the MPSP scheduling algorithm. Assuming the day divided in T=24 equal time slots ∆ , each load profile represents the vector of total energy consumptions of user over the time slots ∆ . This vector
can be written as:
denotes the daily price of energy for time slot where specified by the utility company. In a game theory approach it is assumed that each player is rational, so it aims to maximize its utility function by assuming the best strategy. Therefore in the DSM game each user
chooses the load profile that represents the solution of the following problem: (11)
•
is the pattern of electrical consumption for the electrical appliance that performs the task TSj,
The MAS schedules the tasks TSj choosing accordance to the minimization of two fitness functions , defined by (15) and (16).
The combination of best strategies and of the corresponding values of utility functions constitutes the dominant strategy equilibrium for the game. This dominant strategy equilibrium is the Nash equilibrium [12-13].
T
f1 = ∑α t l t
,
,
,
(15)
t =1
In fact the vector is a Nash equilibrium for the game , , if the following relation is valid: ,
in and
(16)
(12)
The Nash equilibrium for the DSM game exists since the utility function is defined to be concave with respect to . It can be observed that the Nash equilibrium for the DSM game ensures the reduction of PAR since the term in the payoff function encourages the user to select those strategies for which the slot-time-energydemand is more close to the average total time-slot-load.
Let
…
be the vector of the execution time.
If the MAS shifts all patterns of electrical consumption in accordance with X then the corresponding vector defines the total load profile of end-user u . ,… ,… So in (15) and (16) of end-user u.
denotes the total load for each time slot ∑
III.
MPSO SCHEDULING
One MAS of DSM framework controls the load profile of each end-user u of the power grid. The MAS knows exactly the electrical needs of the daily tasks TSj planned by the user. Let {TSj, j=1..ki} be the set of daily tasks to be done, each task is characterized by two vector[14-15] : (13)
(14) where •
is the electric power required to perform the task,
(17)
The function in (15) is the daily electricity cost for the user and denotes the daily price of energy for time slot. The Multi Agent System solves the scheduling problem with a heuristic approach running a MPSO algorithm. Several approaches are introduced in literature to handle multi-objective optimization problems with MPSO. In general a MPSO algorithm mimics the social behavior of a bird flock finding the food. Each individual moves in a multidimensional solution space and is characterized by a position (Xi) and velocity (Vi). In this work the position represents the suggested solution of the scheduling optimization problem while the velocity determines the rate of change of the current position. The MPSO method considered in this paper follows the work of [16] implementing the concepts of Pareto dominance and mutation, explained in the following. In a Multi Objective (MO) optimization problem, a solution X1 is said to dominate the other solution X2 if both the following condition are true:
•
dj is the duration of the task TSj,
•
•
the solution X1 is no worse than X2 in all objectives function values;
tpj is the start time preferred by the user to run the task,
•
the solution X1 is strictly better than X2 in at list one objective function value.
•
tsj and tfj define the maximum time window admitted of task TSj, to run the task; the time of execution satisfies the constrain
;
If a solution X is not dominated by any other solution then it is said to be Pareto optimal. The set of all non-dominated solutions of MO optimization problem constitute the Pareto optimal set and the corresponding values of objective function constitute the Pareto front.
The basic idea of the MOPSO method inn [16] is the use of an external repository in which the Pareto flight experiences are stored. The external archive is employed to t identify a leader to guide the search. The repository has a gridd structure defined in the space of objective function values (thee objective space). This grid divides the search space in equal hyper-cubes h whose dimension is the number of the objective fuunctions. The nondominated solutions can be located in one off these hyper-cubes depending on the values of the objective funcctions. The finite memory of the archive allows the t storage of only a finite number of solutions. So, when the reppository is full and a new non-dominated solution is found in current c swarm, the new solution is compared with the ones in the repository. If the new solution dominates some elements of the archive, such elements are removed and the new solutionn is added. If the repository elements and new solution are all a non-dominated, the number of solution in each hyper-cube is counted and an element is randomly removed from the hyper-cube more crowded. This idea is formalized in the mainn steps of MO-PSO Algorithm, that follows. MO PSO ALGORITHM Initialize population and the external repository by: - Setting randomly the initial positions of each individuual of the swarm; - Setting to zero the initial velocity of the swarm; - Computing the position (i.e. the fitness values) of o each individual in objective space; - Archiving the non dominated-solutions; - Generating the hyper-cubes of the explored search- space by the division of the objective axes in K equal intervals; While the maximum iteration or ideal fitness is not attained a do : - Evaluate the velocity of each individual, Vi, and new n position with the expressions
(18) - If the new position falls out the admissible regions for f the solution space, locate it on the boundary of the admissible regions andd reverse the direction of the velocity; - Apply the mutation operator to each individual; - Evaluate the new positions in objective space andd update the external repository with non-dominated solutions; - If the new position falls out the admissible regions for f the solution space, locate it on the boundary of the admissible regions andd reverse the direction of the velocity; - Apply the mutation operator to each individual; - Evaluate the new positions in objective space andd update the external repository with non-dominated solutions
• the element in position r of Xi is mutated generating a random value if ß>1/hh with h be the number of the iteration IV.
SIMULATION RESULTS
In this section, some experrimental results for the proposed DSM architecture are shown.. The algorithms are tested in Matlab platform on a MacPro Intel I 2x 2.4GHz Quad-Core Intel Xeon with 16 Gb of RAM. It is considered a power grid of N end user with 2 4 the improvement of the
PAR is much more consistent as it is coonsidered an high number of the Pareto solutions for the MPSO O scheduling.
sources. In this case the MP PSO algorithm can be used to determine the scheduling of battery b discharge and also the amounts of energy provided by each storage devices.
RENCES REFER [1]
[2] [3]
[4]
[5] Fig. 7. Electrical demand of the area when the 4 userss run MPSO algorithm but don’t play the game [6]
[7]
[8]
[9]
[10]
[11]
Fig. 8. Electrical demand in the area when the 4 users u run only MPSO algorithm
[12] [13] [14]
V.
CONCLUSION
This paper proposes a Multi Agent fram mework for energy management in MV power grid based onn the concept of distributed optimization. A non-cooperative game is formulated to model the interactions betweeen user’s energy load profiles and shave, at the Nash Equilibrium, the Peak-toAverage Ratio in the geographical area pow wered by the grid reducing also the energy payments to utiliity company. The proposed DSM framework requires a lim mited exchange of information flowing only between the useer and the energy provider, lightening the signaling burden and requiring a simplified communication infrastructure to integrate i in power grid. Besides the user privacy is safeguarrded, because the utility doesn’t directly control the schedulingg of electrical tasks of the users. The DSM framework is welll suited to control users equipped with batteries to store energgy from renewable
[15]
[16] [17]
[18]
NIST Framework and Roadm map for Smart Grid Interoperbility Standards,2012,[Online].Availabble:http://www.nist.gov/smartgrid/uploa d/NIST_Framework_Release_2-00_corr.pdf H.Farhangi, "The path of the sm mart grid," Power and Energy Magazine, IEEE , vol.8, no.1, pp.18,28, Januuary-February 2010 F.Katiraei, R.Iravani, N.Hattziargyriou, A.Dimeas, "Microgrids management," Power and Eneergy Magazine, IEEE , vol.6, no.3, pp.54,65, May-June 2008 J. Medina, N. Muller, and I. Roytelman, “Demand response and distribution grid operations: Opportunities O and challenges,” IEEE Transactions on Smart Grid, vol. 1, no. 2, pp. 193 –198, Sep. 2010. US DoE, “Benefits of demandd response in electricity markets and recommendations for achieving them,” t Report to the US Congress., Feb. 2006. Available at "http://eetd.idii.gov" P.Palensky and D.Dietrich, “D Demand Side Management :Demand Response, Intelligent Energy Systems, and Smart Loads”, Industrial Informatics, IEEE Transactions on, o vol.7, no.3, pp.381-388, Aug.2011. G. Z. Zhu, J. Tang, S. Lambothaaran, W. Chin, and Z. Fan, “An integer linear programming and game theory t based optimization for demandside management in smart grid,” in Proc. IEEE GLOBECOM, 2011. g theory model for the power Afshin Salehian, “Dynamic game transmission grid”, in IEEE PES P Conference on Power Systems Conference and Exposition, Octoober 2004, pp.1449.1452. Amir-Hamed Mohhsenian-Rad, Vincent V W.S. Wong, Juri Jatsevich, and Robert Schober, “Optimal andd autonomous incentive-based energy consumption scheduling algoritthm for smart grid,” in IEEE PES Conference on Innovative Sm mart Grid Technologies,Gaithersburg, Maryland,January 2010, pp.1-6. M.Fahrioglu, MFern, and F.Avarrado, “Designing cost effective demand management contracts using gam me theory”, in Proc. IEEE Power Energy Society Winter Meeting, New Yoork, Jan.1999, pp.427-432. C.Ibars, M.Navarro, and L.Giuppponi, “Distributed demand management in smart grid with a congestionn game”, in Proc. Int.Conf.Smart Grid Communications, Gaithersburg, MD, M Oct.210,pp.397-402. Samson Lasaulce and Hamidou Tembine, “Game Theory and Learning for Wireless Networks”, Elselvier 2011. D. Fudenberg and J.Tirole, “G Game Theory”. Cambridge, MA:MIT Press,1991. S.Salinas, Li Ming Li and Li Paan Li, "Multi-Objective Optimal Energy Consumption Scheduling in Smart Grids", Smart Grid, IEEE Transactions on , vol.4, no.1, pp.3341,348, March 2013 C.Yan, L.Ren Ping, W.Chen, M. M de Groot and Z.Zhiyuan, "Consumer Operational Comfort Level baseed power demand management in the smart grid," Innovative Smart Grid G Technologies (ISGT Europe), 2012 3rd IEEE PES International Connference and Exhibition on , vol., no., pp.1,6, 14-17 Oct. 2012 C.A.C.Coello, G.T.Pulido and M.S. Lechuga, "Handling multiple Computation, IEEE Transactionss on , vol.8, no.3, pp. 256- 279, June D. Zhang, L. G. Papageorgiou, N. J. Samsatli, and N. Shah, “Optimal scheduling of smart homes eneergy consumption with microgrid,” in Proceedings of ENERGY 2011, The First International Conference on Smart Grids, Green Comm munications and IT Energy-aware Technologies. Mestre, Italy, 20111. M. Young, The Technical Writer’s W Handbook. Mill Valley, CA: University Science, 1989.
