Real-time Optimal Energy and Reserve Management of ... - IEEE Xplore

0 downloads 0 Views 828KB Size Report
local sources and sets energy/reserve prices for EVs to increase its revenue, with the ... Index Terms—Fast charging station, regulation reserve, game theory ...
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < ancillary services from EVs has not been considered. It should be mentioned that the flexibility of EVs’ charging processes can provide valuable ancillary services for power systems, e.g., regulation reserves [6-8], spinning reserves [7, 12] and territory reserve [13]. Consequently, the objective of this paper is to manage energy and reserve to benefit both EVs and the aggregator using game theory. When managing energy and reserve of FCSs, the aggregation should consider both the technical and economic characteristics of reserves. Conventional aggregation methods for providing reserves have two requirements: i) EVs are able to inject power back to power systems (V2G), and ii) EVs are connected to the charging/discharging infrastructures for a long period to meet the grid codes related to ancillary services [6-7]. For instance, the spinning reserve should be maintained for 60 minutes [14]. Nevertheless, EVs should be well charged within a given duration, e.g., 30 minutes. From the perspective of the time conflict, V2G is infeasible to provide spinning or territory reserves in FCSs. In addition, prices of spinning and territory reserves are much lower than those of regulation reserves. This indicates that EVs can obtain higher revenues by providing regulation reserves. In this work, EVs are aggregated to provide regulation reserves through the energy and reserve management of FCSs, which is based on the Stackelberg game. In this game, the FCS operator is the leader and EVs are followers. The FCS operator manages local sources and sets energy/reserve prices for EVs to increase its revenue, and EVs determine their charging and reserve strategies. Unlike the aggregation methods in [6]-[7], this paper not only considers the energy losses caused by providing reserves, which is not stated in [6], but also compensates these losses, which is not considered in [7]. Afterword, the bi-level optimization problem is applied to compute the Stackelberg equilibriums (SEs) of the game, and the existence of SEs is proved accordingly. Simulation study based on both single and multiple periods is carried out to verify the merits of proposed management strategy, in comparison with the centralized management method. The main contributions of our work can be summarized as follows. i) The optimal energy and reserve management is modeled as a Stackelberg game, which can benefit the FCS operator and EVs simultaneously. ii) To provide regulation reserves and compensate the energy losses of EVs, novel decision models are proposed for EVs and FCS operator. The rest of this paper is organized as follows. Section II presents the system models. In Section III, the interaction between EVs and the FCS operator is modeled as a Stackelberg game. In Section IV, the property of proposed game is discussed and the solving method is given. Simulation results are analyzed in Section V and conclusions are drawn in Section VI, respectively. II. SYSTEM MODELS A. Diagram of Fast Charging Station Considering a FCS located in highway rest zones [18], an integrated framework [19] is adopted to study the real-time

2

energy and reserve management of the FCS, as shown in Fig. 1. This framework mainly consists of three components, i.e., fast charging piles, energy storage system (ESS) and photovoltaic (PV), which are owned by the FCS operator. Based on the reserve provision states, charging EVs can be categorized into the following types. i) EVs charge and provide reserves for FCS during the operation period, i.e., [t, t+5], t is the operation time, denoted by set C. ii) EVs only charge and do not provide reserves for FCS during the operation period, denoted by set L. Wholesale market Energy Flow Cash Flow Reserve Flow Fast Charging Station

Electric Vehicles

Energy Storage System

Photovoltaic Generations

EVs EVs charging and charging providing reserves

Fig. 1 Framework of a fast charging station. The FCS operator aggregates its own sources, i.e., ESS and PV, and EVs to participate into the wholesale market. The FCS operator charges the energy consumed by EVs and pays for the reserves supplied by EVs.

It is noted that EVs  CL are not considered in this paper, e.g., EVs are not connected to charging piles. Each EV user’s preference parameters, shown in Section III.B, are stored in its battery management system and can be accessed by the FCS operator. It sets energy price and reserve prices for EVs according to energy and regulation reserve prices in the wholesale market to maximize FCS’s revenue. The FCS operator acquires all information within the charging station. Individual EV’s information is invisible to other EVs, and it competes with other EVs. B. Energy and Reserve Market As an EV stays in the FCS for a limited period, this paper focuses on the real-time market. Two different dispatching methods are widely applied in energy and reserve markets, i.e., joint and sequential dispatch [20]. In the sequential approach, the clearing of energy and reserve is separated and sequential. For the joint approach, the clearing of energy and reserve is dealt with simultaneously. Joint dispatch of energy and reserve is deployed in this paper to obtain higher market efficiency [20]. The real-time market clears per 5 minutes [21]. To avoid price spikes, the wholesale market operator publishes ex-ante real time price at 5 minutes time step [21]. The FCS acts as a price taker in the real-time market. The control time of FCS is divided into equal time interval with time step t, where t is set to be 5 minutes. FCS should implement the real-time energy and reserve management every

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 5 minutes, and the information flow is depicted in Fig. 2. ISO 1) Energy price Wholesale 2) Reserve prices during [t,t+60] Market

EVs   L

EVsC

1) Output of PV during [t,t+60] 1)Energy demand 2)Initial energy stored 3)Preference parameters

FCS

1)Duration of stay 2)Energy demand 3)Initial energy stored 4)Preference parameters

t-5 t: Operation time

FCS

1)Energy price during [t,t+5] EVs   L

B. Decision Model for EVs Charging

Frequency PV regulation Real signal output

Real-time charging adjustment

1)Energy price 2)Reserve prices during [t,t+5]

EVsC

EVs

C

Time t+5

t

t+60

            

PV

between the FCS and power systems, and prices for EVs, satisfying constraints (4b)-(4y).

ISO

Regulation reserves during [t,t+5]

3

operation horizon of FCS

Fig. 2 Schematic representation of the information flow. Before t-5, the FCS operator obtains ex-ante energy/reserve prices from the wholesale market, EVs’ information and PV output within the station. During [t-5, t], the FCS operator sets energy and reserve prices for each EV during [t, t+5]. Before t, the FCS operator submits regulation reserves to the independent system operator (ISO). During operation horizon [t, t +5], ISO sends regulation signals to the FCS, and the FCS operator adjusts charging rates of EVsC.

With the assumption that the FCS can be controlled by the independent system operator (ISO) directly, it is feasible for the FCS to submit its reserve plans to ISO before operation time t, as shown in Fig. 2. Besides, the real-time energy and reserve management must be finished by operation time t. The regulation reserves provided by the FCS should be maintained in [t, t+5]. When the quantity of regulation reserves cannot meet the amount cleared, i.e., reserve deficiency, the FCS will be punished. III. NON-COOPERATIVE STACKELBERG GAME

For an EVL, a utility function UEVL,i(xi, x-i; ai, bi, Ce,i, E0,i) is introduced to capture its utility, regarding to the energy price Ce,i set by the FCS operator. xi is the i-th player’s strategy, i.e., charging plan of the i-th EV, denoted by PEV,i. x-i=[x1, …,xi-1 , xi+1,…, xN], which represents other players’ strategies. Preference parameter ai is the maximal price of the i-th EV willing to pay for per unit of energy. Preference parameter bi is the price sensitive parameter, which depends on EV user’s attitude towards the energy price. For instance, if the 1-th EV is more sensitive to the energy price than the 2-th EV, e.g., the 1-th EV’s user is more frugal, b1>b2 and the 1-th EV consumes less energy than the 2-th EV when the energy prices are the same for both EVs. E0,i is the initial energy stored in the i-th EV. The charging plan may vary based on the price willing to pay and the price sensitive parameter of each EV. The price per unit of energy can also affect the charging plan of EVsL. The utility level is also influenced by the initial energy stored in EVsL. Inspired by [9], properties are described as follows, which the utility function of an EVL must satisfy. i) The utility function of an EVL is considered to be non-decreasing, as each EVL is interested in consuming more energy unless it reaches its maximum energy demand level.

U EVL,i ( xi , x i ; ai , bi , Ce,i , E0,i ) PEV ,i

 0, i   L

(1a)

A. Game Formulation To study the real-time energy and reserve management of a FCS, a Stackelberg game is introduced. It is a powerful tool for analyzing multi-level decision-making process, where a cluster of independent followers react to the decision taken by one or more leaders [22]. In the real-time management, the FCS operator is the leader and EVs are followers. This game is defined in normal form,  = {P, S, U}. P is the set of players, and P ={CLF}, where F stands for the FCS operator. S is an N-tuple of pure strategy sets, one for each player , i.e., S={Si}iP. N is the number of player in game . N=NL+NC+1, where NL is number of EVsL and NC is the number of EVsC. U is an N-tuple of utility functions, i.e., U={Ui}iP. The proposed game has the following decision models. i) Utility function of each EVL (2a) captures the utility by consuming energy. The strategy space Si, iL corresponds to its charging plan, satisfying constraint (2b). iii) Utility function (3a) of each EVC represents the benefit of consuming energy and providing reserves. Strategy space Si, iC corresponds to the charging and reserves plan, satisfying constraints (3b)-(3g). iv) Utility function (4a) for the FCS operator depicts the total profits that it can receive by trading energy and reserves between EVs and power systems. The strategy space SF corresponds to the amount of energy/reserves exchange

ii) The marginal utility of an EVL is considered as a decreasing function, as the level of utility gradually gets saturated when more energy is consumed.  2U EVL,i ( xi , x i ; ai , bi , Ce ,i , E0,i ) 2 PEV, i

 0, i   L

(1b)

iii) For a fixed charging plan PEV,i, a larger ai implies higher utility, and a larger bi leads to a lower utility.

U EVL,i ( xi , xi ; ai , bi , Ce,i , E0,i ) ai U EVL,i ( xi , xi ; ai , bi , Ce,i , E0,i ) bi

 0, i  L

(1c)

 0, i   L

(1d)

iv) The price for per unit of energy set by the FCS operator affects the utility of an EV and the utility decreases with a higher price.

U EVL,i ( xi , x i ; ai , bi , Ce,i , E0,i ) Ce,i

 0, i   L

(1e)

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
0 and bi>0. In addition, the demand curve of an EVL can be depicted by Fig.3, when (2a) is applied to capture its utility by consuming energy.

Demand curve of EV i

Energy price ($/kWh)

(0, ai)

(Ei, ai  bi Ei )

0

E0

Ei

Emax,i Energy demand(kWh)

Fig. 3 Energy demand curve of the i-th EV L. To buy one unit energy, i.e., 1kWh, the maximal price for the i-th EV L willing to pay is ai.

The strategy space of i-th EV SEVL,i, iL includes the following constraint. 0  PEV,i  Pmax , Emin,i  Ei  E0,i  PEV,ic t  Emax,i , i   L (2b)

where Emin,i and Emax,i are the minimal and maximal energy requirement of the i-th EV, respectively. Pmax is the maximal charging capacity of the charging pile. E0,i and Ei are energy stored in the i-th EV’s battery at t and t+5, respectively. c is the charging efficiency of the charging pile. The charging power and energy limitation are given in (2b), respectively. As the charging plan of each EVL should satisfy constraint (2b), the benefit of an EVL might be negative when Emin,i=Ei.

U EVC,i ( xi , x i ; ai , bi , Ce,i , Cru,i , Crd,i , E0,i )

1 1  ai Ei  bi Ei2  ai E0,i  bi E0,2 i 2 2 U D Ce,i PEV,i t  Cru,i REV,  i t  Crd, i REV, i t , i   C

(3a)

where RUEV,i and RDEV,i are the up and down reserves provided by the i-th EVC, respectively. Cru,i and Crd,i are the up and down reserve prices for the i-th EVC, which are set by FCS operator, respectively. To meet EV’s energy requirement and power constraints when providing reserves, the following constraints should be included into the strategy space of the i-th EVC. U Emin,i  E0,i  PmaxcTr,i  ( PEV,i  REV, i )c t , i   C

(3b)

)c t  Emax,i -E0,i , i  C

(3c)

( PEV,i  R

D EV, i

D PEV,i  REV, i  Pmax , i   C

(3d)

0  PEV,i  R

(3e)

U EV, i

U EV, i

R (Emax,i, ai  bi Emax,i)

4

D EV, i

,R

, i  C

 0,i  C

Ei  E0,i  PEV,ic t , i  C

(3f) (3g)

where Tr,i is remaining duration for the i-th EV before leaving the FCS. The minimal energy requirement is guaranteed by (3b), considering the energy loss due to the provision of up reserve. When the i-th EV C provides down reserve, the energy obtained cannot exceed the maximal energy requirement, as shown in (3c). The up and down reserves provided by the i-th EV are limited as shown in (3d) - (3f). As explained in (2b), constraint (3b) forces EVs to charge their battery to meet their minimal energy requirements. The benefit of an EVC might be negative when Emin,i- E0,i -PmaxcTr,i = (PEV,i- RUEV,i)ct. D. Decision model of the FCS operator For the FCS operator, it can sell/buy energy from the real-time market, sell regulation reserves to the wholesale market and energy to EVs to increase its profit, while paying for regulation reserves of EVs and reserve deficiency. The utility function of the FCS operator, UF (xF, x-F; CE, CRU, CRD, PEV,i, RUEV,i , RDEV,i), is described as follows.

C. Decision Model for EVs Charging and Providing Reserve For an EVC, it can maximize its utility by not only optimizing the charging plan but also providing up and down reserves for the FCS, corresponding to the energy and reserve prices set by the FCS operator. The utility function for an EVC, UEVC,i(xi, x-i ; ai, bi, Ce,i, Cru,i, Crd,i, E0,i), is given as follows.

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < max

U D { PFCS ( t ), RFCS ( t ), RFCS ( t ), U PESS,dc ( t ), PESS,c ( t ), RESS ( t ), D U RESS ( t ), BESS ( t ), RESS,c ( t ), U D D RESS,dc ( t ), RESS,c ( t ), RESS,dc ( t ),  U ( t ), D ( t ), Cp ,i , Cru ,i , Crd ,i }S F

U D U F ( xF , x F ; CE , CRU , CRD , PEV,i , REV, i , REV, i )

5

charging rate, when BESS(t)=0, as shown in (4g). Equations (4h) and (4i) show the maximal down reserve provided in the charging range and discharging range, respectively. They are interpreted in the similar way as those of (4f) and (4g).

T

U D  { [ CE (t ) PFCS (t )  CRU (t ) RFCS (t )  CRD (t ) RFCS (t ) t 1

Benefit by selling energy

Charging

PESS,c,max

- CESS ( PESS,dc (t )  PESS,c (t ))  Cpenalty ( U (t )   D (t ))] Battery degradation cost





iC L

Discharging range Charging range

Charging

Benefit by selling regulation reserves

R (t )R D ESS

Cost of reserve deficiency



Ce,i PEV,i

 (C

iC

Benefit by selling energy to EVsC  L

ru, i

U D REV, i  Crd, i REV, i )}t

Cost of buying reserves from EVsC

(4a) where CE(t), CRU(t) and CRD(t) are the real-time energy, regulation up and down reserve price in time slot t, respectively. CESS is the degradation cost of ESS. T is the operation horizon. The degradation loss model of ESS is taken from [23]. To manage the ESS and EVs systematically, the operation strategy of FCS operator should meet following three types of constraints, i.e., these constraints are integrated into SF.

D ESS,c

PESS,c,max

  (t )  

   D (t ) RESS,c      D RESS (t )  

PESS,c (t )

 

U (t ) RESS,c      U  RESS (t )      U RESS,dc (t )  

PESS,dc (t )

PESS,dc,max

PESS,dc,max

  

Discharging

BESS (t )  1

    D RESS,dc (t )  

  U U RESS,dc (t ) RESS (t )  



BESS (t )  0

Discharging

a) ESS is charging

b) ESS is discharging

Fig. 4 Relationship between set-point and reserves of ESS.

i) Power and energy constraints of ESS 0  PESS,c (t )  BESS (t ) PESS,c,max , t  T

(4b)

0  PESS,dc (t )  (1  BESS (t )) PESS,dc,max (t ), t  T

(4c) EESS,min (t )  EESS (t  t ) 

U PESS,dc (t )  RESS,dc (t )

(4e)

D EESS (t  t )  ( PESS,c  RESS,c )ESS,c t  EESS,max

(4f)

EESS (T )  EESS,0

0R

U ESS,dc

(t )  PESS,c (t ), t  T

U 0  RESS,dc (t )  PESS,dc,max  (1  BESS (t )) PESS,dc (t ), t  T (4g) D 0  RESS,dc (t )  PESS,dc (t ), t  T D ESS,c

(t )  PESS,c,max  BESS (t ) PESS,c (t ), t  T

(4h) (4i)

where PESS,c(t) and PESS,dc (t) are the charging and discharging rates of ESS in time slot t, respectively. BESS(t) is a binary variable, indicating whether ESS is charging or not, i.e., when ESS is charging, BESS(t) =1. PESS,c,max and PESS,dc,max are the maximum charging and discharging rates of ESS, respectively. RUESS(t) and RDESS(t) are the up and down reserves provided by ESS in time slot t, respectively. R UESS,c (t) and R UESS,dc (t) are the up reserves provided by ESS in the charging range and discharging range, respectively. RDESS,c(t) and RDESS,dc(t) are the down reserves provided by ESS in the charging and discharging ranges, respectively. T is the set of operation time slots. Equations (4b)-(4c) indicate that ESS can only be charging or discharging during each time slot. The up and down regulation reserves of ESS can be provided when ESS is charging or discharging, as shown in (4d)-(4e) and Fig. 4. In this figure, when ESS is charging, i.e., BESS(t)=1, the maximal up reserve in charging range cannot exceed the charging rate of ESS, as shown in (4f). The maximal up reserve provide in the discharging range is limited to the maximal charging rate, when BESS(t)=1, or the gap between the discharging rate and maximal

ESS,dc

t (4j)

t

D D D RESS (t )  RESS,c (t )  RESS,dc (t ), t  T

U ESS,c

0R

ESS,dc

(4d)

R

U ESS,c

(t )  R

PESS,dc (t )

(t ), t  T

U ESS

(t )  R

EESS (t )  EESS (t  t )  PESS,c (t )ESS,c t 

(4k) (4l) (4m)

where EESS(t) is the energy stored in ESS by time t. ESS,c and ESS,dc are charging and discharging efficiency of ESS. EESS,0 is initial energy stored in ESS. EESS,min and EESS,max are the lower and upper boundaries for energy stored in ESS, respectively. Equation (4j) shows the dynamic energy state change of ESS. The maximal energy loss is formulated in (4k) due to provision de up reserve when ESS is discharging. Equation (4l) manifests the maximal energy surplus. By the end of operational horizon, the energy state of ESS is the same as the initial energy state, shown in (4m). ii) System level constraints of the FCS PFCS (1)  PPV (1) 



iC L

PEV,i  PESS,dc (1)  PESS,c (1) (4n)

PFCS (t )  PPV (t )  PESS,dc (t )  PESS,c (t ), t  T \{1} PFCS (t )  R

U FCS

(t )  PFCS,max , t  T

PFCS,min  PFCS (t )  R

D FCS

 (1)  PPV,max (1)  PPV (1)  R U

U FCS

(1) 

R

iC

D  D (1)   PPV,min (1)  PPV (1)  RFCS (1) 

(4p)

(t ), t  T U EV,i

R

(4o) (4q)

R

U ESS

(1)

(4r)

D  RESS (1)

(4s)

U U  U (t )  PPV,max (t )  PPV (t )  RFCS (t )  RESS (t ), t  T \ {1}

(4t)

iC

D EV,i

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < Power exchange between FCS and the grid (kW)

25 20

Utility ($)

15 10

EVs utility FCS operator utility

5

Social welfare 0 -5 -10

0.09 0.07 Cp,max($/kWh)

0.11

300

-200

-700

0:00

In this study, the price cap constraint (4w) plays an important role in the benefit distribution among players, as shown in Fig.6. With the increase of Cp,max, social welfare remains unchanged, the benefit of FCS operator rises, while the EV users’ benefit decreases drastically. Therefore, Cp,max can be used to regulate the benefit allocation among the EVs and FCS in practical application. In Fig. 7, the impacts of power exchange limitation between the FCS and power systems are studied. With the increase of the minimal power exchange limitation between FCS and power systems, i.e., PFCS,min, the social welfare and utility of FCS operator decreases accordingly, while the utility of EVs remains the same. This is becasuse the power exchange limitation (4q) is managed by the FCS operator. It sells less down reserves to power system, when PFCS,min increases. 20

15 Utility($)

Uncontrolled charging Proposed method

800

-1200

0.05

Fig. 6 Benefits allocation among FCS operator and EVs with respect to Cp,max. Cp,max is the upper boundary of energy price set by FCS operator.

EVs utility

10

FCS operator utility Social welfare 5

0 -800

9

-700

-600

-500 PFCS,min (kW)

-400

-300

Fig. 7 Benefits allocation among FCS operator and EVs with respect to PFCS,min.

4:10

8:20

12:30 Time

16:40

20:50 23:55

Fig. 8 Power exchange between FCS and power systems in each time slot.

In Fig.8, the power exchange plan between FCS and power systems along the simulation day is shown. The uncontrolled charging leads to heavy load to power systems during [4:30, 4:35] and [19:50, 19:55], and the proposed method can eliminate these peaks. The up and down regulation reserves provided by the FCS in each time slot is shown in Fig. 9. Compared with the uncontrolled charging, FCS can provide 2.04 times of up regulation reserve, i.e., 4480.4kWh in uncontrolled charging and 13598.64 kWh in the proposed method, as illustrated by Fig. 9 a). The reason is that the proposed method can stimulate EVs’ provision of up reserve by adopting (4x). In addition, as shown in Fig. 9, FCS provides almost the same amount of down reserves in both methods, i.e., 4579.5 kWh in uncontrolled charging and 4682.9 kWh in the proposed method. Moreover, the power exchange of the proposed method, including reserves shown in Fig. 9, is within [-800kW, 600kW], which demonstrates the proposed method can avoid heavy load to power systems. In Fig. 10, typical energy profiles of four EVs are studied to check whether there exist energy losses due to providing reserves. It should be noted that 289 EVs arrive at the FCS. These EVs have difference initial energy stored in their batteries. For ease of presentation, we select four representative energy profiles. It is observed that the energy stored in each EV is 70kWh after 30 minutes charging, which indicates these EVs are well charged. Consequently, the provision of reserve does not lead to energy losses for EVs during their charging processes.

C. Results of Multi-periods Simulation

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 1500

1000

Utility($)

It is seen from Fig.8 to Fig.13, the proposed method can increase the welfare of each EV, FCS operator and social, simultaneously, without loss of energy for each EV. Moreover, the social welfares in CM and proposed method are the same, i.e., 869.37$. This verifies the merit of the proposed game based method, i.e., balancing benefits between EVs and FCS in the aggregation of EVs and FCS to provide regulation reserves.

Social welfare 500

FCS utility EV utility

14 Proposed method Uncontrolled charging

0

10 -500 5%

8

10% 15% 20% 25% Uncertainty level of PV

6

Note: Uncertainty level of PV =1-

30%

PPV,min PPV,max

4

Fig. 14 Impacts of PV uncertainty level on each player’s utility.

2

It is noted that the wholesale market settlement is assumed to be ex-ante and the uncertainty of prices (CE, CRU, CRD) is not considered. When the wholesale market settlement is ex-post, uncertainty related to these prices exists. The technique used to manage the price uncertainty is given in Appendix E.

0 0:00

4:10

8:20

12:30 Time a)

16:40

20:50

23:55

14 Proposed method Uncontrolled charging

12 10 8 6 4

D. Computational Efficiency Analysis The computational efficiency, e.g., computational time, is analyzed, under different number of EVs, operation horizon and tightness. The simulation parameters are the same as those of the single-period simulation, as shown in Section V. A. It is noted the searching strategy of CPLEX is set to be the branch and cut method, and the cuts generation strategy is clique and disjunctive cuts.

2

16:40

20:50

23:55

Fig. 13 Utilities of social welfare and FCS operator in each time slot. a) Social welfare. b) Utility of FCS operator.

The impact of uncertainty level of PV on each player’s utility is studied in Fig. 14. According to the review of minutes-ahead and hourly-ahead accuracy presented in [37], the uncertainty level is studied within [5%, 30%], which can be guaranteed by most of the forecasting techniques. With the increase of uncertainty level, the social welfare and utility of FCS operator decrease accordingly. In addition, as the uncertainty is managed by the FCS operator, the rise of uncertainty level deteriorates the utility of FCS operator, while the utility of EVs remains almost the same, as shown in Fig. 14.

150 100 50 0 15 10

5

le

12:30 Time b)

Sca

8:20

Computational time(s)

200

4:10

Vs  C

0 0:00

of E

Social welfare ($)

12

Utility of FCS operator($)

11

[t,t+25]

[t,t+100] [t,t+75] [t,t+50] Operation horizon

Fig. 15 Effects of number of EVs and operation horizon on computational efficiency. Tightness gap is set to be 10-5. System level constraints (4p)-(4q) are omitted.

In Fig. 15, only the number of EVsC is taken into consideration, since KKT conditions shown in Appendix D introduce more binary variables and decision variables, compared with KKT conditions of EVsL. As shown in Fig. 15, the computational time increases with the increasing scale of EVs, when the operation horizon is within [t+5 minutes, t+125 minutes]. Moreover, the computational time increases remarkably when the scale of EVs is higher than 9. Furthermore, Fig. 15 shows that the expansion of operation horizon does not

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < affect the computational efficiency as remarkably as that of scales of EVs. In Fig. 16, the computational time is reduced evidently, if the tightness gap is set to be 10-1, problem (8) under different scales of EVs can be solved within 1 second.

To manage uncertainties of PV and reserve called, the FCS operator will adjust the output of ESS, charging of EVsC to avoid reserve deficiency, as shown in (10c). max

Computational time(s)

20

15

10

5

0

2

4

6 8 10 Scale of EVs C

min

U  U ( t ), D ( t ), rESS ( t ), rcalled ( t ), pPV ( t ) D U D rESS ( t ), rEV ,i ( t ), rEV ,i ( t )

Tightness gap set to 105 Tightness gap set to 104 Tightness gap set to103 Tightness gap set to 102 Tightness gap set to 101

12

14

Fig. 16 Effects of number of EVs and tightness on computational efficiency. The operation horizon is set to be [t, t+60]. The system level constraints (4p)-(4q) are omitted.

VI. CONCLUSION In this paper, a Stackelberg game based real-time energy and reserve management method has been proposed for a FCS, equipped with ESS and PV. FCS operator aggregates its own sources and EVs to participate in jointed energy and reserve market. It decides energy and reserve prices for EVs to maximize its benefit whereas EV users manage their charging and reserves plans to maximize their own utilities. The hierarchical game is reformulated as an MIQP problem and the existence of SEs has been proved. Simulation results have shown that the proposed management method can avoid heavy load to power systems. Benefits of EVs and FCS operator increase along with the increase of social welfare when EVs provide reserves for the FCS. Impacts of uncertainty on the benefit of FCS operator and EVs have been studied as well. APPENDIX A. Uncertainty management of the FCS When managing energy and reserve during the operation horizon, i.e., [t, t+60] in this paper, we consider the uncertainty of PV output and reserve called by ISO in each time slot. The uncertainty management strongly depends on the forecasting information. The interval forecasting method is used to forecast the short-term output of PV [37]. Therefore, the uncertainty of short-term PV output is expressed by the following set PV. PV ={pPV (t )|PPV,min (t )  PPV (t )  pPV (t )  PPV,max (t )  PPV (t )}, t  T

12

U F ,real  -  [ U (t )+ D (t )] tT

D D D min{0, rcalled (1)  pPV (1)}    rEV, i  rESS (1)   (1)  iC  U U U r  r (1)   (1)  max{0, rcalled (1)  pPV (1)}   EV,i ESS iC  min{0, r (t )  p (t )}  r D (t )   D (t ), t   \{1} called PV ESS T  U U rESS (t )   (t )  max{0, rcalled (t )  pPV (t )}, t  T \{1} (10c)  U U s.t. 0  rESS (t )  RESS (t )  D D 0  rESS (t )  RESS (t ) 0  r U  R U EV, i EV, i  D D 0  rEV, i  REV, i  U  (t ),  D (t )  0  

where rUEV,i and rDEV,i are the up and down reserve employments of the i-th EV C, respectively. rUESS(t) and rDESS(t) are the up and down reserve employments of ESS during time slot t, respectively. UF,real is the function of reserve deficiency. It is noted that (10c) is a robust optimization (RO) problem. By introducing affine relationship between the decision variables, i.e., rUEV,i, rDEV,i , rUESS(t) and rDESS(t), and uncertainty factors, i.e., pPV(t), rcalled(t), this RO problem is treated as the affine RO (ARO) problem [38]. When the output of PV and regulation reserve called by ISO reveal themselves during [t, t+5], reserve employments of ESS and EVsC are assumed to follow the formulations with respect to uncertainty factors, as shown in (10d)-(10g). U rESS (1) 



iC U rEV, i 

U RESS (1) max{0, rcalled (1)  pPV (1)} U U REV,i  RESS (1)

R

U EV, i

iC

D rESS (1)  

U REV, i U  RESS (1)

(10d)

max{0, rcalled (1)  pPV (1)} (10e)

D RESS (1) min{0, rcalled (1)  pPV (1)} (10f) D D (1)  REV,i  RESS

iC D rEV, i 

R

iC

D REV, i

D EV, i

D  RESS (1)

min{0, rcalled (1)  pPV (1)} (10g)

(10a) where pPV(t) is the derivation from forecasting value PPV(t). For regulation reserve called by ISO, its uncertainty can be depicted by the following set R. D U R ={rcalled (t ) |  RFCS (t )  rcalled (t )  RFCS (t )}

(10b)

EVs C and ESS are over-charged when condition (10h) hold. EVs C are over-deferred and ESS is over-discharged when condition (10i) holds. D D (10h) min{0, rcalled (1)  pPV (1)}    REV, i  RESS (1) iC

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)
0, the decision model (2) and (3) are concave quadratic programming problems with respect to the prices, i.e., Ce,i, Cru,i, Crd,i. The concave quadratic programming problem has the only global optimal solution [39], which completes the proof. Based on proposition 1, the solution of each EV’ decision model is unique to the prices set by FCS operator , which means xiPi(xF) , iLC is singleton, where Pi(xF) is the reaction set of the i-th EV for fixed strategy of FCS operator. Bi-level optimization problem (6) can be expressed as follows. max U F ( xF ,  i 1

i  N L  NC

xF S F

 E

 E

where  +E ,  -E , L and U are the corresponding Lagrange multipliers for constraint (2b). B+E , B-E, BL and BU are binary variables for complementary constraints. M is a big scalar. Equation (12a) is KKT condition for decision model (2). Equations (12b) - (12i) are complementary constraints for the Lagrange multiplier with each constraint and have been further reformulated as mix-integer linear constraints based on the big-M reformulation technique [27]. D. Karush-Kuhn-Tucker Conditions for Decision Model (3) By introducing Lagrange multipliers for each constraint, KKT conditions for model (3) can be represented as follows. (ai  bi Ei  Ce,i )c t -E+c t +E-c t +R+ -R- -P+  0

(13b)

Crd,i t  E-c t  R+     0

(13c)

0 B M + E

( PEV,i  R

U EV, i

Pi ( xF ))

(11)

+ E

(13d)

)c t  Emin,i  E0,i  PmaxcTr ,i  (1  B ) M (13e) + E

0  E-  BE- M Emax,i  E0,i  ( PEV,i  R

D EV, i

(13f)

)c t  (1  B ) M E

C. Karush-Kuhn-Tucker Conditions for Decision Model (2) The KKT conditions for decision model (2) are shown as follows.

E+c t  Ec t  L  U  0, i   L

(12a)

 (1  B ) M R

U EV, i

R

 (1  B ) M +

 (1  B ) M +

0 B M 

D EV, i

R

(13i) (13j)

R

0 B M +

(13g) (13h)

+ R

0 B M R

U EV, i

From the FCS operator's perspective, (11) can be viewed as a mathematical program with an implicitly defined constraint region given by the EV’s decision model [25]. Optimal solutions, including optimal prices, of (11) obtain the maximal FCS utility and capture the unique strategy of each EV responding to the optimal prices. This optimal solution meets condition (5) for all players of the game , which completes the proof of Theorem 1.

-aic t  bi Eic t  Ce,i t

Pmax  PEV,i  R

D EV,i

PEV,i  R

i

(13a)

Cru,i t  E+c t  R     0

0  R+  BR+ M

s.t. xi  arg max U i ( xi , xF ), i   L  C xi 

+ E

 E

When either of the conditions (10g)-(10h) holds, in order to guarantee the security of ESS and energy requirements of EVs, the FCS cannot provide the amount of reserve cleared, resulting in the reserve inadequacy, as shown in (4r) and (4s). During [t+5, t+60], the regulation reserves are only provided by ESS. It is overcharged or over-discharged when the following conditions hold.

13





 (1  B ) M

(13k) (13l) (13m) (13n) (13o)

0  P+  BP+ M

(13p)

0  PEV,i  (1  B ) M

(13q)

+ P

where +E , -E, +R , -R, +, - and +P are Lagrange multiplier corresponding to each constraint, B+E , B-E, B+R , B-R, B+, B- and B+P are binary variables for complementary constraints. Equations (13a) - (13c) are KKT conditions for decision model (3). Equations (13d) - (13q) are complementary constraints with big-M reformulation.

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < E. Robust Decision Models to Manage Price Uncertainties When uncertainties of energy and reserve prices are considered, e.g., in ex-post pricing, the uncertainty management is not the same as that of ARO as shown in Appendix A. Omitting the correlation among these prices, the energy prices and regulation reserves prices are formulated by the following interval sets. CE ={CE (t ) | CE,min (t )  CE (t )  CE,max (t )}

(14a)

CRU ={CRU (t ) | CRU,min (t )  CRU (t )  CRU,max (t )} (14b) CRD ={CRD (t ) | CRD,min (t )  CRD (t )  CRD,max (t )} (14c)

where CE,min(t) and CE,max(t) are the lower and upper forecasting prices of energy prices in time slot t , respectively. CRU,min(t) and CRU,max(t) are the lower and upper forecasting prices of up reserve prices in time slot t, respectively. CRD,min(t) and CRD,max(t) are the lower and upper forecasting prices of down reserve prices in time slot t , respectively. Based on the explicit maximization method in [7], (4a) can be reformulated as the following robustness counterpart.

[9].

[10].

[11].

[12].

[13].

[14].

[15].

[16].

[17].

U D max U F ( xF , x F , CE , CRU , CRD , PEV,i , REV, i , REV, i ) xF S F

T

 { [CE,max (t )  PFCS (t )   CE,min (t )  PFCS (t )  

[18].



t 1

U D CRU,min (t ) RFCS (t )  CRD,min (t ) RFCS (t )

(14d)

-CESS ( PESS,dc (t )  PESS,c (t ))  Cpenalty ( U (t )   D (t ))] 



iC  L

Cp,i PEV,i 

 (C

iC

ru, i

U D REV, i  Crd, i REV, i )}t

[19].

[20].

[21].

where [x]+ and [x]- are functions of max(x,0) and min(x,0) respectively. REFERENCES [1].

[2].

[3].

[4].

[5].

[6].

[7].

[8].

D. P. Tuttle, and B. Ross ,“Technological, Market and Policy Drivers of Emerging Trends in the Diffusion of Plug-in Electric Vehicles in the US.” The Electricity Journal, vol.28, no.7 pp. 29-43, Aug.2015. W. Lee, L. Xiang, R. Schober, and V. W. S. Wong, “Analysis of the behavior of electric vehicle charging stations with renewable generations.” Proc. IEEE Conf. Smart Grid Commun., Vancouver, Canada, Oct. 2013. K. Yunus, H. Z. De La Parra, and M. Reza, “Distribution grid impact of plug-in electric vehicles charging at fast charging stations using stochastic charging model.” Proc. 14th Eur. Conf. Power Electron. Appl. (EPE), Birmingham, U.K., Aug. 2011. F. Banez-Chicharro , J. M. Latorre, and A. Ramos, “Smart charging profiles for electric vehicles.” Computational Management Science, vol.11, no.1, pp.87-110, July 2014. N. Shahraki, H. Cai, M. Turkay, and M. Xu, “Optimal locations of electric public charging stations using real world vehicle travel patterns.” Transportation Research Part D: Transport and Environment, vol.41, pp.165-176, Dec. 2015. A.Y.S. Lam, K.C. Leung, V.O.K Li. "Capacity management of vehicle-to-grid system for power regulation services." 2012 IEEE Third International Conference on Smart Grid Communications (SmartGridComm), Tainan, Taiwan , Nov. 2012. X. Bai, W. Qiao. "Robust optimization for bidirectional dispatch coordination of large-scale V2G." IEEE Trans. Smart Grid ,vol.6, no.4,pp :1944-1954, Jul. 2015. M. Shafie-khah, E. Heydarian-Forushani, M.E.H. Golshan. “Optimal

[22]. [23].

[24]. [25]. [26].

[27].

[28].

[29].

[30].

[31]. [32]. [33].

14

trading of plug-in electric vehicle aggregation agents in a market environment for sustainability. ” Applied Energy, vol.16, no.2, pp: 601-612, Jan.2016. W. Tushar, W. Saad, H. V. Poor, and D. B. Smith, “Economics of electric vehicle charging: A game theoretic approach.” IEEE Trans. Smart Grid, vol.3, no.4, pp. 1767-1778, Dec. 2012. S. Bahrami, M. Parniani, “Game theoretic based charging strategy for plug-in hybrid electric vehicles.” IEEE Trans. Smart Grid, vol.5, no.5, pp. 2368-2375, Sept. 2014. W. Lee, X. Lin, R. Schober, and V.W.S. Wong, “Electric Vehicle Charging Stations With Renewable Power Generators: A Game Theoretical Analysis.” IEEE Trans. Smart Grid, vol.6, no.2, pp. 608-617, March 2015. S. Cheng, Y. C. Hsu, C. H. Gan, and Y. B. Lin, “Charge scheduling of electric vehicles in highways.” Mathematical and Computer Modelling, vol.57, no.11, pp.2873-2882, June 2013. J.M. Foster, G. Trevino, M. Kuss, and M.C. Caramanis, “Plug-in electric vehicle and voltage support for distributed solar: theory and application.” IEEE Systems Journal, vol.7, no.4, pp. 881-888, Dec. 2013. E. Erik, M. Milligan, and B. Kirby. “Operating reserves and variable generation.” National Renewable Energy Laboratory, CO, Rep.TP-5500-51978, Aug. 2011. M. Kintner-Meyer, “Regulatory policy and markets for energy storage in North America.” Proceedings of the IEEE, vol.102, no.7, pp.1065-1072, Jul. 2014. D. Bhatnagar, A. Currier, J. Hernandez, O. Ma, and B. Kirby. Market and policy barriers to energy storage deployment. Sandia National Laboratories, CA, Rep.SAND2013-7606, Sep.2013. J. P. Torreglosa, P. Garcí a-Triviño, L. M. Fernández-Ramirez, and F.Jurado, “Decentralized energy management strategy based on predictive controllers for a medium voltage direct current photovoltaic electric vehicle charging station.” Energy Conversion and Management, vol.108, pp.1-13, Jan. 2016. Tesla, Supercharger. [Online]. Available at https://www.tesla.com/supercharger N Liu, Q Chen, X Lu, J Liu, et al. “A charging strategy for PV-based battery switch stations considering service availability and self-consumption of PV energy.” IEEE Trans. on Industrial Electronics, vol.62, no.8, pp: 4878-4889, Aug. 2015. P. González, J. Villar, C.A. Dí az, F.A. Campos. “Joint energy and reserve markets: Current implementations and modeling trends.” Electric Power Systems Research, vol.109, pp: 101-111, Apr.2014. Q. Wang, C. Zhang, Y. Ding, G. Xydis, , J. Wang, and J. Østergaard, “Review of real-time electricity markets for integrating Distributed Energy Resources and Demand Response.” Applied Energy, vol.138, pp. 695-706, Jan. 2015. R.B. Myerson. Game theory. Harvard university press, 2013. Q. Jiang, M. Xue, and G. Geng. “Energy management of microgrid in grid-connected and stand-alone modes.” IEEE trans. power systems ,vol.28,no.3, pp: 3380-3389, Aug.2013. T. Başar and G. J. Olsder, Dynamic Noncooperative Game Theory.Philadelphia, PA: SIAM, 1999. Bard, Jonathan F. Practical bilevel optimization: algorithms and applications. Springer Science & Business Media, 2013. J. Outrata, M. Kocvara, and J. Zowe, Nonsmooth approach to optimization problems with equilibrium constraints: theory, applications and numerical results. Springer, 1998. S. Siddiqui and S. A. Gabriel, “An SOS1-based approach for solving MPECs with a natural gas market application,” Netw. Spat. Econ., vol. 13, no. 2, pp. 205–227, Jun. 2013. C. Bliek1ú, B. Pierre, and L. Andrea, “Solving mixed-Integer quadratic programming problems with IBM-CPLEX: a progress report.” Proceedings of the Twenty-Sixth RAMP Symposium, Tokyo, Japan, Oct. 2014. B. Christian, and B. Pierre, Solving Mixed-Integer Quadratic Programming problems, [Online]. Available at: http://www.orsj.or.jp/ramp/2014/paper/4-3.pdf O Stein, P Kirst, and P Steuermann. “An Enhanced Spatial Branch-and-Bound Method in Global Optimization with Nonconvex Constraints. ” Optimization Online (2013). H. P. Williams, Model Building in Mathematical Programming, 4-th ed. New York, NY, USA: Wiley, 1999. Electric Power Monthly, [Online]. Available: https://www.eia.gov/electricity/monthly H. Quan, D. Srinivasan, and A. Khosravi, “Short-term load and wind

1949-3053 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSG.2017.2687522, IEEE Transactions on Smart Grid

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT)