Stackelberg Game based Energy and Reserve ... - IEEE Xplore

Stackelberg Game Based Energy and Reserve Management for a Fast Electric Vehicle Charging Station Tianyang Zhao, Energy research institute @NTU Nanyang Technology University, Singapore [email protected]

Xuewei Pan School of Mechanical Engineering and Automation, Harbin Institute of Technology (Shen Zhen)

Abstract— The problem of energy and reserve exchange between electric vehicles (EVs) and fast charging station (FCS) is studied using a Stackelberg game. In this game, the FCS operator, who acts as a leader, needs to set its energy and reserve prices to optimize its revenue while ensuring EV users’ charging demand. On the other hand, EV users, who act as the followers, needs to decide their charging and reserve strategies to optimize a tradeoff between the benefit from battery charging and reserves provision. It is shown that the proposed game possesses a social optimal Stackelberg equilibrium, in which the FCS operator optimizes its prices and the EV users choose their equilibrium strategies. A mathematical programming with equilibrium constraints (MPEC) reformulation of the game is proposed and can be solved efficiently by commercial software packages. The reformulation enables the FCS operator and EV users to reach the equilibrium and is assessed by extensive simulations. Keywords—fast charging station, regulation reserve, Stackelberg game

I. INTRODUCTION With increasing penetration of electric vehicles (EVs), more charging infrastructures are required to meet the energy requirement of EVs [1]. With large charging capacities, FCSs can significantly shorten the charging duration. FCSs can be directly integrated into transmission networks, which make their sittings more flexible. Thus, charging at FCS is one of the promising methods to stimulate the deployment of EVs [2]. With cooperatively development, FCSs and EVs have the potential to participate in the wholesale market. However, charging at FCS still faces some inevitable challenges, like adverse impacts on power systems [3] and high initial investments [4]. These problems can be tackled from the planning and operation of FCSs. Optimal sitting and sizing is an effective method to make the planning of FCSs more economical. The objectives are to minimize the total costs of FCSs and distribution network losses and maximize the vehicle-milestraveled [5]. When operating an FCS, which has its own energy source, such as PV and ESS, FCS operator seeks to maximize its revenue through optimally managing its energy exchange with power system and providing ancillary services, while

directly or indirectly controlling EVs’ charging process [6]. When charging EVs at FCS, EV users pursuit comfortable services and less cost through strategically consuming energy by adjusting the integration time [8]. The management of FCS should balance the benefits of FCS operator and EV users. A powerful tool to balance interests between multiple entities is game theory. Game theory is firstly applied to manage large scale EVs charging in [8]. The energy benefit interaction between the grid and EV users is modeled as a Stackelberg game, where the grid acts as the leader and EV users act as the followers [8]. The authors of [9] propose an game theory based EV charging strategy, where the EV users try to minimize charging costs by adjusting the integration time. The price competition among multiple EV charging stations is modeled as a super modular game in [10]. Aforementioned game theory based works [8]-[10] only focus on energy-benefit aspects. However, through proper coordination, the controllability of EV charging can provide valuable ancillary services (i.e. spinning reserve and territory reserve [7] [11]). The conventional approaches of providing ancillary services by controlling EVs charging all require: 1) EVs can inject power back to power systems (V2G) and 2) EVs are connected to the charging/discharging infrastructure for a long period of time to meet the grid codes of ancillary services. For example, the spinning reserve should be maintained for 60min [12]. But when charged at FCSs, EVs should be well charged within a given time, i.e. 30min [2]. The time limitation makes V2G uneconomical and technically infeasible to provide spinning/territory reserve. According to the pay as performance rule [13], the revenue of providing spinning/territory reserves is low [12]. Therefore, there are both market and technical barriers for FCSs to benefit from providing ancillary services. With the development of deregulation, capacity limited facilities, i.e. ESS, nowadays have the chance to provide regulation reserve [12]. When EVs are charging at FCSs, the FCS operator can aggregate the capacity limited EVs to increase its capacity. Consequently, it is possible for FCS operator to aggregate EVs charging at FCSs to provide regulation reserves. As a key aspect of this kind of aggregation, balancing the benefits between the FCS operator and EV users hasn’t been extensively studied.

This work was supported by project Adaptive Integrated Hybrid DC-AC Micro Power Parks System (M4094016) of the Energy Research Institute, Nanyang Technological University (ERI@N), Singapore. Paper no. TEC00505-2015.

978-1-5090-2998-3/17/$31.00 ©2017 IEEE

Shuhan Yao, Peng Wang School of electrical and electronic engineering, Nanyang Technology University, Singapore

1417

From a technical perspective, charging at FCSs would cause heavy load, resulting in under voltage or other adverse impacts on power systems [14]. ESS or PV is employed in FCS to mitigate these impacts. Furthermore, in real-time market, the uncertainty factors still exist, like the regulation reserve called by power systems, the output of PVs and so on. ESS can make the operation more reliable and flexible. Since the energy and reserve are strongly correlated, how to make the real-time management of the FCS more reliable while benefiting EV users and the FCS simultaneously is a challenging work. This paper focuses on the energy and reserve exchange between an FCS and EVs, where the FCS can participate in the real-time energy and reserve market. The main contribution of our work can be summarized as follows: 1) energy and reserve exchange between the FCS and EVs is formulated as a Stackelberg game, which enables EVs providing regulation reserve; 2) a novel decision model for EVs is proposed to strategically manage energy and reserve; 3) In particular, the existence of an efficient Stackelberg equilibrium is proved. The rest of this paper is organized as follows. The system models are described in Section 2. In Section 3, the interaction between EVs and the FCS is modeled as a non-cooperative Stackelberg game. In Section 4, the property of the Stackelberg game is discussed and solving method is given. Numerical results are analyzed in Section 5 and conclusions are drawn in Section 6. II. SYSTEM MODELS A. Fast Charging Station

Fig. 1. Framework of the FCS

To study interactions between FCS and EVs, we propose an integrated FCS framework, as shown in Fig.1. The FCS framework mainly consists of three components: fast charging piles, ESS and PV, which are all owned by FCS operator and equipped with controllers. Based on whether providing reserves or not, EVs can be categorized into two types: i) EVs provide reserves to the FCS during the reserve provision period ([t, t+5], as shown in Figure 2), denoted by the set ΩC. ii) EVs do not provide reserves to the FCS during reserve provision period, denoted by the set ΩL. Each EV user’s preference parameters (see Section 3.2 and Section 3.3) are stored in its battery management system (BMS) and can be accessed by the FCS. The FCS operator sets energy price and reserve prices for the EVs according to energy and regulation reserve prices in wholesale market to maximize

FCS’s revenue. Individual EV’s information is invisible to each other, and there is no cooperation among EVs. B. Energy and Reserve Market Framework

­ ° ° ® ° ° ¯ ­ ° ° ® ° ° ¯ Fig. 2. Schematic representation of the information flow

Since EVs can only stay in FCS for a limited period of time, we only focus on the real-time market. There are two different dispatching methods commonly used in energy and reserve markets: joint and sequential dispatch. Jointed dispatch of energy and regulation reserve and is deployed in this paper to obtain higher efficiency [15]. The real-time market clears per 5 minutes [15]. To avoid price spikes, the wholesale market operator would publish the ex-ante real time price at 5-minutes time step [15], same to the real-time market employed in NYISO, as shown in Figure 2. The FCS acts as a price taker in the real-time market. The control time of FCS is divided into equal time interval with time step Δt, where Δt is set to 5 minutes. FCS implements the management every 5 minutes and the information flow can be depicted by Figure 2. With the assumption that the FCS can be controlled by the independent system operator (ISO) directly, it is feasible for the FCS to submit its energy exchange and reserve plan to the wholesale market before operation, as shown in Figure 2. Besides, the real-time energy and reserve management must be finished during [t-5, t], where t is the time of operation. The regulation reserves provided by the FCS should be maintained in the next 5 minutes [t, t+5]. The FCS operator would be punished if the quantity of regulation reserve can not meet the amount cleared. III. NON-COOPERATIVE STACKELBERG GAME A. Game Formulation Stackelberg game, which is a type of non-cooperative game deals with the multi-level decision making process of a number of independent decision makers or players (the followers) in response to the decision taken by a leading player (the leader) [16]. Hence, we formulate a non-cooperative Stackelberg game in which the FCS operator is the leader and the EV users are the followers. This game is defined in its normal form, Γ = {(, +, -}. ( is the set of players, (ΩC∪ΩL∪), where 
sands for the FCS operator; + is N-tuple of pure strategy sets; - is Ntuple of payoff functions, N=NC+NL+1, NC is the number of

1418

EVs∈ΩC, NL is the number of EVs∈ΩL. Our Stackelberg game has the following components: i) There are two types of EVs, EVs∈ΩC and EVs∈ΩL, in which act as the followers in the game and respond to the price set by the FCS, and the FCS act as the leader. ii) The strategy space +i, i∈ΩL corresponds to the amount of energy demanded from the FCS, satisfying the constraints (1a)-(1b). The utility function of each EVs∈ΩL (1c) captures the benefit of consuming the demanded energy. iii) The strategy space +i, i∈ΩC corresponds to the amount of energy demanded from the FCS and reserves to the FCS, satisfying the constraints (3a)-(3f). The utility function (3g) of each EVs∈ΩC captures the benefit of consuming the demanded energy and providing reserves. iv) The strategy space +i, i∈ corresponds to the amount of energy and regulation reserves exchange between the FCS and the grid, satisfying the constraints (5a)-(5g) and (5l)-(5n). The utility function (5k) for the FCS operator captures the total profits that FCS operator can receive by trading energy between EVs and the grid.

assumption in microeconomics, EV i's utility gradually gets saturated as more energy is consumed with preference parameter bi>0 [17]. And the benefit of EV i is shown in the shadow area of Figure 3 when paying the energy by price pe,i. It is further assumed that aiηc-biEmax,iηc>CE, where CE is the wholesale market energy price. Consequently, the utility function of EV i∈ΩL, -i can be depicted as following:

B. Decision model for an EV∈ΩL For an EV∈ΩL, it can maximize its benefit by optimizing the charging strategy. The strategy space of EV i∈ΩL, +i should include the following constraints: (1a) Ei ≤ Emax,i , 0 ≤ PEV ,i ≤ Pmax , ∀i ∈ Ω L Ei = E0,i + PEV ,iηc Δt , ∀i ∈ Ω L

(1b)

where PEV,i is the charging strategy of EV i, Emax,i is maximum energy demand by EV i, Pmax is maximum charging capacity of the charging pile, E0,i and Ei are energy stored in EV i’s battery at t and t+5 respectively, ηc is charging efficiency of the charging pile. The energy limitation and charging power limitation is given in (1a), respectively. Equation (1b) depicts the relationship between energy stored in EV i's battery and its charging strategy.

1 1 max -i = ai Ei − bi Ei2 − ai E0,i + bi E0,2 i 2 2 − pi PEV ,i Δt , ∀i ∈ Ω L

PEV ,i ∈+i

Proposition 1: When EV i∈ΩL is charging, PEV,i>0, the energy price for EV i∈ΩL pe,i would not exceed aiηc-biEiηc. Proof: The KKT conditions for decision model (1) are shown as following: -aiηc Δt + bi Eiηc Δt + pe,i Δt + λE,iηc Δt − λL,i + λU ,i = 0, ∀i ∈ΩL (2a)

λE ,i ( E0,i + PEV ,iη c Δt − Emax,i ) = 0, ∀i ∈ Ω L

(2b)

λL ,i PEV ,i = 0, ∀i ∈ Ω L

(2c)

λU ,i ( PEV ,i − Pmax ) = 0, ∀i ∈ Ω L

(2d)

λE ,i , λL ,i , λU ,i ≥ 0, ∀i ∈ Ω L

(2e)

where λE,i, λL,i and λU,i are the corresponding Lagrange multipliers for constraint (1a). Based on KKT condition shown in (2a), if pe,i>aiηc-biEiηc, then λL,i >0, resulting in PEV,i=0, which is contradict to the premise. The proof completes. C. Decision model for EV∈ΩC For EV∈ΩC, it can maximize its benefit by not only optimizing the charging plan but also providing up/down reserve to the FCS. Strategy space of EV i∈ΩC, +i should include the following constraints: U Emin,i − E0,i − PmaxηcTr ,i ≤ (PEV ,i − REV ,i )ηc Δtˈ( λEU ,i ) ∀i ∈ΩC (3a) D ( PEV , i + REV , i )η c Δ t ≤ E max, i - E 0, iˈ( λ ED , i ) ∀ i ∈ Ω C

0 ≤ PEV , i − R R

U EV , i

,R

U EV , i

,( λ PU , i ) ∀ i ∈ Ω C

Fig. 3. Energy demand curve of EV i

For EV i∈ΩL, we define a utility function, -i (xi, x-i, ai ,bi) to represents the level of satisfaction that an EV∈ΩL obtains as a function of the energy it consumes, where xi represents for EV i' strategy and x-i=[x1, x2, …xi-1,xi+1,…, xN-1, xN]. Here ai is the maximum price EV user i would like to pay for per unit of energy. According to no free lunch rule, ai should be bigger than 0. Preference parameter bi define the slope of EV user i's demand curve, as shown is Figure 3. Followed by the

(3c)

≤ Pmaxˈ( λPD ,i ) ∀i ∈ Ω C

(3d)

D EV , i

≥ 0,( λU ,i , λ D , i ) ∀ i ∈ Ω C

(3e)

Ei = E0,i + PEV ,iηc Δt , ∀i ∈ ΩC

ai − bi Emax,i

(3b)

D EV , i

PEV ,i + R ai − bi Ei

ai − bi Emax,i

(1c)

(3f)

where RUEV,i and RDEV,i are the up and down reserves provided by EV i∈ΩC, Emin,i is the minimal energy demand of EV i, Tr,i is remaining time for EV i before leaving the FCS, and λEU,i, λED,i , λPU,i , λPD,i , λU,i and λD,i are the Lagrange multipliers for each constraint. When EV i∈ΩC provides reserves, its energy demand should be guaranteed within the given range as shown in (3a) (3b). The up/down reserves provided by EV i should be limited as shown in (3c) - (3e). For EV i∈ΩC, we define a different utility function, -i (xi, x-i, ai ,bi) to represents the level of satisfaction that an EV∈ΩL obtains as a function of the energy it consumes and reserves it provides. The utility function of EV i∈ΩC, -i can be depicted as following:

1419

1 1 -i = ai Ei − bi Ei2 − ai E0,i + bi E0,2 i − pe,i PEV ,i Δt {PEV ,i , Rev ,i , Rev,i }∈Si 2 2 (3g) U D , + pru,i REV Δ t + p R Δ t ∀ i ∈Ω ,i rd ,i EV ,i C

U PPV − PPV ,min + RFCS ≤

max U D

where pru,i and prd,i are up and down reserve prices payed by the FCS operator. Proposition 2: When the down reserve prices prd,i >0, PEV0,i is no bigger than PEV,i +RDEV,i , where PEV0,i is charging rate of EV i ∈ΩC when it does not provide reserves. Proof: By introducing Lagrange multipliers for each constraint as shown in (3a)-(3g), KKT conditions for model (3) can be represented as follows: -aiη c Δt + bi Eiη c Δt + pe ,i Δt − λEU ,iη c Δt (4a) + λED ,iη c Δt + λPD ,i − λPU ,i = 0ˈ∀i ∈ Ω C − pru ,i Δt + λEU ,iη c Δt + λPU ,i − λU ,i = 0ˈ∀i ∈ ΩC

(4b)

− prd ,i Δt + λED ,iη c Δt + λPD ,i − λD ,i = 0ˈ∀i ∈ ΩC

(4c)

U λEU ,i [Emin,i − E0,i − PmaxηcTr ,i − (PEV ,i − REV ,i )ηc Δt ] = 0ˈ∀i ∈ΩC (4d)

D (4e) λED ,i [( PEV ,i + REV , i )η c Δt − Emax,i + E0, i ] =0ˈ ∀i ∈ Ω C U λPU ,i ( REV ,i − PEV , i ) = 0ˈ∀i ∈ Ω C

λPD ,i ( PEV ,i + R

D EV ,i

(4f)

− Pmax ) = 0ˈ∀i ∈ ΩC

(4g)

λU ,i R

= 0ˈ∀i ∈ Ω C

(4h)

λD , i R

= 0ˈ∀i ∈ ΩC

(4i)

U EV ,i

D EV , i

λEU ,i , λED ,i , λPD ,i , λPU ,i , λU ,i , λD ,i ≥ 0ˈ∀i ∈ Ω C

(4j)

Equation (4a)-(4c) represents the KKT conditions for PEV,i , R and R DEV,i, respectively. The complimentary relationships between the Lagrange multipliers and their corresponding constrains are shown in (4d)-(4i). As shown in (4c), when prd,i >0 and λD,i≥0, λED,i>0 or λD,i>0. When λED,i>0, constraint (3b) is active, which results in PEV0,i < PEV,i +R DEV,i . When λD,i >0, PEV,i +R DEV,i =Pmax and apparently PEV0,i < PEV,i +RDEV,i. The proof completes.

D PPV ,max − PPV + RFCS ≤

i∈ΩC ∪ΩL

0≤R

0 ≤ PESS ,c ≤ PESS , c ,max

(5b)

0 ≤ PESS , dc ≤ PESS , dc ,max

(5c)

U ESS

≤ PESS , dc ,max +PESS , c -PESS , dc

(5d)

D ESS

≤ PESS , c ,max +PESS , dc -PESS , c

(5e)

0≤ R

EESS ,min ≤ EESS ,0 −

U ESS

PESS , dc − PESS , c + R

η ESS , dc

Δt

D E ESS ,0 + ( PESS , c − PESS , dc + RESS )η ESS , c Δ t ≤ E ESS ,max

U FCS

R

D FCS

,R

≥0

(5f) (5g) (5h)

U EV , i

U + RESS

(5i)

¦R

D EV , i

D + RESS

(5g)

i∈ΩC

where PFCS is set point of energy exchange between FCS and the grid, PPV is the forecasting value of PV, PESS,c and PESS,dc are charging and discharging rates of ESS, PESS,c,max and PESS,dc,max are maximum charging and discharging rates of ESS, RUESS and RUESS are up reserves provided by ESS, ηESS,c and ηESS,dc are charging and discharging efficiency of ESS, EESS,0 is initial energy stored in ESS, EESS,min and EESS,max are minimum and maximum energy stored in ESS, RUFCS and RDFCS are up and down regulation reserves provided by FCS, PPV,min and PPV,max are the minimum and maximum forecasting output of PV during the reserve provision period. Equation (5a) depicts power balance within the FCS. Maximum charging and discharging capacity of ESS are given in (5b) - (5c). The capacities of up and down reserves of ESS are shown in (5d)-(5e). To avoid over-charging of ESS, up and down reserve provided by ESS should meet energy limitation shown in (5f)-(5g). Equation (5h) shows that regulation reserve can only positive reserve capacity. To guarantee the reserves robustness of reserves within the FCS, when the output of PV and regulation reserve called by the grid both call for up and down reserve, there should be sufficient up and down reserve capacities as shown in (5i)-(5g). The utility function of  , - is depicted as follows:

max

D U D {PFCS , RU FCS , RFCS , PESS ,dc , PESS ,c , RESS , RESS , pe , pru , prd }∈S

U EV,i

D. Decision model for FCS operator For the FCS operator, it can maximize its profits through energy and reserve trading between the grid and EVs, while optimally mange its own sources, i.e. ESS and PV. The strategy space of , + should include the following constraints: PFCS = PPV − ¦ PEV ,i + PESS , dc − PESS , c (5a)

¦R

i∈ΩC

-

U D = CE PFCS Δt + CRU RFCS Δt + CRD RFCS Δt-CESS (PESS ,dc + PESS ,c )Δt (5k)

+

¦

i∈ΩC ∪ΩL

U D pe,i PEV ,i Δt − ¦ ( pru,i REV ,i + prd ,i REV ,i )Δt t∈ΩC

where CRU and CRD are up and down regulation reserve prices in the wholesale market, CESS is cost parameters of ESS charging and discharging [18]. As shown in (5k),  can maximize its profits through selling/buying energy to/from the grid, selling regulation up/down reserve to the grid, selling energy to EVs and

buying reserves from EVs.

bi ( Pηc Δt ) 2 . 2 Proof: According to Proposition 1, when EV i∈ΩL is charging pe,i ≤aiηc-biEmax,iηc Since  aims to maximize its own revenue, the optimal price for EV i is aiηc-biEiηc. Thus, b the utility EV i∈Ω is i ( PEV ,iηc Δt ) 2 . 2 When EV i∈ΩL is not charging, i.e., PEV,i=0, this proposition holds naturally, which completes the proof. Proposition 4: For EV i∈ΩC, the maximum utility is bi D 2 [( PEV ,i + REV , i )η c Δt ] , when it does not provide reserves. 2 Proof: For EV i∈ΩC, when it can not provide reserves, the b utility obtained by charging is i ( PEV 0,iηc Δt ) 2 .It can be proved 2 through a similar way as Proposition 3. According to

1420

Proposition 3: For EV i∈ΩL, its utility is

Proposition 2, the maximum charging rate of EV i∈ΩC is PEV,i +RDEV,i, when it does not provide reserves. The maximum utility b D 2 is i [( PEV ,i + REV , i )η c Δt ] , which completes the proof. 2 To guarantee the reserve provision of EVs i∈ΩC, according to the marginal rate of transformation [17] rule in microeconomics, the following constraint should be added to +, as follows: U aiη c -bi ( Ei -REV , iη c Δ t )η c -p e , i = p ru , i , ∀ i ∈ Ω C

(5l)

aiη c -bi Ei η c -pe , i = p rd ,i , ∀ i ∈ Ω C

(5m)

pe ,i ≤ aiη c -bi ( E0,i + Pmaxη c Δt )η c , ∀i ∈ ΩC

(5n)

When EV i∈ΩC is providing up reserve (it might lower its charging rate during the reserve provision period), it is likely to obtain less energy. The extreme condition for EV i∈ΩC providing up reserve is that it is charging at the up reserve capacity PEV,i-RUEV,i. Under this condition, the marginal increase of EV i’s utility with respect to per unit of charging power can be depicted by left side of the inequality in (5l). When EV i∈ΩC is providing down reserve (it might increase its charging rate during the reserve provision period), it is likely to obtain excess energy. The extreme condition for EV i ∈ΩC providing down reserve is that it is charging at the set point PEV,i. Under this condition, the marginal increasement of EV i’s utility with respect to per unit of charging power can be depicted by left side of the inequality in (5m). Equation (5n) is the price cap limitation for , which guarantees the reserves provision of EV i∈ΩC. Proposition 5: By employing (5l) - (5n), it is guaranteed that EV i∈ΩC can provide reserve to the FCS. Proof: For an EV∈ΩC, it can choose provide reserves to the FCS or just call for charging serves. As shown in Proposition 4, EV i ∈ΩC can at most bi D 2 obtains [( PEV ,i + REV ,i )ηc Δt ] , when it does not provide 2 reserves. When the energy and reserve prices shown in (5l) (5n) are employed, the utility difference of EV i ∈ΩC before and after providing reserves is shown as following: 1 1 U ai Ei − bi Ei2 − ai E0,i + bi E0,2 i − pe,i PEV ,i Δt + pru,i REV ,i Δt 2 2 bi D D 2 [(PEV ,i + REV + prd ,i REV , i Δt − ,i )ηc Δt ] 2 U D = (aiηc − bi E0,iηc -pe,i )( PEV ,i + REV ,i + REV ,i )Δt

1 3 U2 2 2 2 U D U D − bi [ (PEV ,i + REV REV ,i − REV (6) ,i + REV ,i ) − ,i REV ,i ]ηc Δt 2 2 U D ≥ (aiηc − bi E0,iηc -pe,i )(PEV ,i + REV ,i + REV ,i )Δt 1 U D 2 2 2 − bi (PEV ,i + REV ,i + REV ,i ) ηc Δt 2 ≥ 0, ∀i ∈ΩC Equation (6) shows that, when EV i∈ΩC provides reserves to the FCS, it can at least obtain the same benefit when it does not provide reserves, which completes the proof. It should be noted that, since decision model (5) is convex optimization problem respect to FCS operator’s decision

variables, according to the statement in Section II.C of [18], the ESS would not be charged and discharged simultaneously, i.e., PESS,c*PESS,dc=0. IV. EXISTENCE OF STACKELBERG EQUILIBRIUM AND SOLVING METHOD

A. Existence of Stackelberg Equilibrium In noncooperative games, the existence of an equilibrium solution (in pure strategies) is not always guaranteed [19]. Therefore, for our proposed game Γ, we need to investigate the existence of the Stackelberg equilibrium (SE). Definition 1: Consider the Stackelberg game Γ={(, +, -}

defined in Section III-A, where (,
+
and -
are given by (1) ,(3) and (5) respectively. A strategies profile x∗ ∈+ constitutes the SE of game Γ, if and only if it satisfies the following set of inequalities: (7) -i ( xi∗ , x∗− i ) ≥ -i ( xi , x∗− i ), ∀xi ∈ +i , i ∈ ( Theorem 1: For a fixed price, an equilibrium exists among EVs in the proposed game Γ. Proof: The decision model (1) and (3) are quadratic programming problems and independent to other EVs’ strategies. With all bi>0, they are all convex programming. The global optimal solution for each EV is guaranteed [20], which completes the proof. Theorem 2: For fixed charging and reserve plans of EVs, a unique equilibrium for the FCS operator exists in the proposed game Γ. Proof: Decision model (5) is linear programming problem respect to the decision variables of . Thus, a unique strategy exists for the FCS operator, which completes the proof. Theorem 3: A SE of the proposed game exists and is social optimal, when the strategy space of FCS operator is non-empty. Proof: A SE of the proposed game Γ is a strategy profile [ΠNi=1 x*1 ]∈+, where x*N represents for the strategy of . When

+ is non-empty, there exists a unique x *N , according to Theorem 2. Suppose there exists another SE [x1, x2, …,xi,…,x1N-, x*N ] , which means the EVs can obtain higher benefit other than taking strategy x*1 , x*2 , …,x*i , …,x*N-1. It is contradict to Theorem 1. Existence of the SE is proved. As a result, the SE, in which  sets its optimal price in response to the equilibrium demands of the EVs and reserve supplements of EVs, represents the socially optimal solution of the proposed game Γ [8]. The social optimal objective is the sum of all players’ utilities (utilitarian objective). The centralized social optimal management (CM) model for game Γ can be represented as following: max -

1421

x∈+

U D = CE PFCS Δt + CRU RFCS Δt + CRD RFCS Δt -CESS (PESS ,dc + PESS ,c )Δt (8)

1 1 (ai Ei − bi Ei2 − ai E0,i + bi E0,2 i ) 2 2 i∈ΩC ∪ΩL The proof completes. +

¦

B. Solving Method The proposed Stackelberg game Γ is a bi-level optimization problem [21]. The popular method to solve the bi-level optimization problem is reformulation the problem as a mathematical programming with equilibrium constraints (MPEC) [22]. The MPEC problem for our game Γ can be represented as follows: max - x∈S

­(2a) − (2e) (9) ° s.t. ®(4a) − (4 j ) °(5a) − (5 g ), (5l ) − (5n) ¯ Due to the bilinear items pe,iPEV,i, pru,iRUEV,i and prd,iRDEV,i in (5k), the objective function is non-concave. Compliment constraints (2c)-(2e) and (4d)-(4i) are reformulated to mixinteger linear constraints, through big-M reformulation [23]. (9) is mix-integer nonlinear programming (MINLP) problem. Based on spatial branch and bound searching method, IBM CPLEX 12.6 [24] is employed to solve this MINLP problem. V. CASE STUDY

set to (ai -0.06)/Emax,i. Energy and regulation reserve prices in the wholesale market are taken from NYISO [28]. Cp,max is set to 120$/MWh, CE is set to 12.45 $/MWh, and CRD and CRU are set to 8.49 $/MWh and 8.49 $/MWh. In the single period simulation, four scenarios are proposed, as shown in Tablet 1. The forecasting output of PV lies in [500kW, 600kW], and PPV=550kW. In the time sequential simulation, real-time energy and regulation reserve price on 2015 October 20 [28], and intrahour output of PV provided by NREL [29] at the same area were adopted. The arrival of EVs at each time interval is assumed to follow the Poisson distribution, with parameter 1. Reserve called by the grid and output of PVs are assumed to follow the uniform distribution. When the output of PV and reserve called are known in real-time operation, re-dispatching of the ESS and EVs∈ΩC follows (10a)-(10d). U RESS U (10a) rESS = max{0, rcalled + ΔpPV } U U ¦ REV ,i + RESS i∈ΩC

A. Case description To verify the effectiveness of the proposed method, two cases studies have been carried out. In the first case, we focused on the single time period preformation of the proposed method under various scenarios. In the second case, a time sequential simulation has been carried out. In model (8), EVs users pay for the energy by 0.12$/kWh, where 0.12 $/kWh is the mean retail energy price for the residential consumer in US [25]. 
pays for the reserves of EVs by CRD and CRU. In the FCS, Pmax is set to 120kW, ηc is set to 0.95, which is the same to Tesla supercharger [26]. PESS,c,max and PESS,dc,max are set to 200kW and 200kW, with charging efficiency 0.9 and discharging efficiency 0.9, respectively. EESS,min and EESS,max are set to 20kWh and 200kWh. The capacity of PV within the FCS is 600kW, and FCS operator can forecast real time output of PV by interval prediction method [27]. CESS is set to 10 $/MWh.

CMa

2

No

b

SG

3

Yes

CM

4

Yes

SG a.

(10b)

D RESS min{0, rcalled + ΔpPV } D D REV , i + RESS

(10c)

U EV , i

D rESS =−

U + RESS

¦

i∈ΩC

D rEV ,i = −

D REV ,i

¦R

D EV , i

D + RESS

min{0, rcalled + ΔpPV }

(10d)

i∈ΩC

where rcalled is the reserve called by the grid during the reserve provation period, ΔpPV is the output derivation of PV with respect to the forecasting value PPV, rUEV,i and rDEV,i are up and down reserves called of EV i∈ΩC, r UESS and r DESS are up and down reserves called by the FCS of ESS. B. Results of single period simulation TABLE II.

SIMULATION SCENARIOS FOR SINGLE PERIOD SIMULATION EVs can provide Management method Scenario reserve or not No

max{0, rcalled + ΔpPV }

¦R

i∈ΩC

TABLE I.

1

U REV ,i

U rEV ,i =

BENEFITS OF FCS OPERATOR AND EVS UNDER DIFFERENT SCENARIOS

Benefit ($)

FCS operator

EVs∈ΩC

EVs∈ΩL

Social welfare

Scenario 1

10.97

-1.83

-2.74

6.40

Scenario 2

6.20

0.10

0.10

6.40

Scenario 3

10.97

-1.42

-2.74

6.81

Scenario 4

3.63

3.08

0.10

6.81

CM represents for centralized decision model (8) b.

SG represents for Stackelberg game model (9)

There are 5 EVs∈ΩC, and 5 EVs∈ΩL. Emax of each EV is set to 70 kWh, which is the same to the battery size of Tesla model S [26]. Emin of each EV∈ΩC is assumed to be 63kWh. E0 of each EV ∈ΩC is set to 25 kWh. Maximum charging duration time of each EV∈ΩC is assumed to be 30minutes. E0 of each EV∈ΩL is set to 60 kWh. Preference parameters of ai are set to vary within the interval [0.06, 0.12] evenly, and bi is

Results in Tablet 2 show that: 1) both CM method and SG method can increase social welfare to the same level, 2) EVs users can obtain a higher benefit when providing reserves to the FCS, and 3) FCS operator can share its benefit with EV users by using proposed method. Since social welfares, i.e., last column in Tablet 2, are the same in both methods, there is no social welfare loss due to the selfishness of FCS operator and EV users. Social welfare increases by 3.64% when EVs

1422

privation of EVs∈ΩC, as shown in Proposition 5.

U REV

can provide reserves to the FCS. These results demonstrate that our SG based method can obtain a social welfare SE, as proved in Theorem 3.

Fig. 6. Up reserves strategies of EVs∈ΩC under different scenarios Fig. 4. Benefit of EVs∈ΩL under different scenarios

Fig. 7. Benefits allocation among FCS operator and EVs with respect to max (pe,i)

Fig. 5. Benefit of EVs∈ΩC under different scenarios

Similarly, as results shown Figure 4 and Figure 5, EV users obtain more benefits utilizing the SG method with respect to different preference parameters ai. Figure 4 and Figure 5 illustrate that benefits obtained by EV users increase as the preference parameters ai increases under all four scenarios. According to Proposition 1, when EVs cannot provide reserves to the FCS, EVs users would be charged at their marginal prices aiηc-biEiηc in the proposed method (scenario 2 & 4). For each EV, aiηc-biEmax,iηc=60$/MWh, which is higher than the wholesale market energy price CE (12.45 $/MWh), resulting in all EVs are charging at the maximum charging rate, as shown in Figure 6. In our simulation, EV i’s preference parameter bi= (ai - 0.06)/Emax,i and Emax,i for each EV i ∈ΩL are the same (70kWh). According to Proposition 3, their benefits would be linearly related to ai, as shown in Figure 4. When EV users can provide reserves to the FCS (EVs∈ΩC), the results shown in Figure 5 demonstrate that EV users’ benefit would be significantly improved. Their charging and down reserve strategies are the same for all four scenarios, 120kW and 0 kW respectively. According to Proposition 4, the maximal utilities of EVs∈ΩC are same to the benefits under scenario 2, as shown in Figure 5, when they do not provide reserves. When EVs∈ΩC can provide reserves to the FCS (scenario 3 & 4), the up reserve strategies are the same in the CM method and SG method (113.68 kW), as shown in Figure 6. The SG method can increase EV users’ benefit by [22.12, 74.50] times based on the calculation of results under scenario 2 and scenario 4 in Figure 5. This proves by adopting (5l)-(5n), EVs s∈ΩC can be obtain at least the same benefits when they do not provide reserves, guaranteeing reserve

For EVs∈ΩC providing reserves to the FCS, CM method can increase EV users’ benefit by [0.1603, 0.3512] times, through the calculation of results under scenario 1 and scenario 3. These rates are much lower compared with our SG method. This proves that the FCS operator shares more benefit with EV users in the SG method. In our simulation, the price cap constraint (5n) plays an important role in the benefit distribution among players, as shown in Figure 7. With the increase of upper limitation of energy price set by FCS operator, max(pe,i), social welfare remains the same, benefit of FCS operator increases, while the EV users’ benefit decreases drastically. In practical application, the price cap limitation can be used to regulate the benefit allocation among the EV users and FCS. C. Results of time sequential simulation

Fig. 8. Benefits of FCS operator and EV users during the simulation period

The proposed method is implemented during the 8:0012:00. Variations of the benefits of FCS operator and EVs

1423

users with respect to both methods through the simulation are shown in Fig.8. It is easy to find that the social welfare achieved of both methods stay the same (42.98 $) along the simulation period. To accelerate the commercialization of EV, FCS operator are supposed to share its benefit (67.82 $) with EV users. Simulation results of the proposed methods gives us that EV users can share a total benefit 23.61 $ from FCS (67.82 $), which can’t be realized by CM method. This verifies the main merit of the proposed game based method: balancing benefits between EV users and FCS.

[9]

[10]

[11]

[12]

VI. CONCLUSION A non-cooperative Stackelberg game based real-time jointed energy and reserve management method for FCS operator has been proposed in this paper. FCS operator can aggregate EVs’ charging flexibility and its own sources participating into jointed energy and reserve market, while balancing benefits among itself and EVs. In the proposed game, FCS operator, acts as the leader, decides energy and reserve prices for EVs and optimally manage its own sources to maximize its benefit, while sharing its benefits with EV users and guaranteeing the reserve provision of EVs. EV users, act as followers, manage their energy and reserves plans to maximize their own utilities in respond to the prices set by the FCS operator. Existence of the Stackelberg equilibrium between the FCS operator and EV users is proved. The Stackelberg game is reformulated as a non-convex MIQP problem and solved by commercial software package. Simulation results have shown that the proposed method yields improved benefits, in terms of the gains of EVs users providing reserves, while FCS operator can share benefits with EVs. The proposed method can obtain the social equilibrium between the FCS operator and EV users. REFERENCES [1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

D. P. Tuttle, and B. Ross ,“Technological, Market and Policy Drivers of Emerging Trends in the Diffusion of Plug-in Electric Vehicles in the US.” The Electricity Journal, vol.28, no.7 pp. 29-43, Aug.2015. W. Lee, L. Xiang, R. Schober, and V. W. S. Wong, “Analysis of the behavior of electric vehicle charging stations with renewable generations.” Proc. IEEE Conf. Smart Grid Commun., Vancouver, Canada, Oct. 2013. K. Yunus, H. Z. De La Parra, and M. Reza, “Distribution grid impact of plug-in electric vehicles charging at fast charging stations using stochastic charging model.” Proc. 14th Eur. Conf. Power Electron. Appl. (EPE), Birmingham, U.K., Aug. 2011. F. Banez-Chicharro , J. M. Latorre, and A. Ramos, “Smart charging profiles for electric vehicles.” Computational Management Science, vol.11, no.1, pp.87-110, July 2014. N. Shahraki, H. Cai, M. Turkay, and M. Xu, “Optimal locations of electric public charging stations using real world vehicle travel patterns.” Transportation Research Part D: Transport and Environment, vol.41, pp.165-176, Dec. 2015. P. Fan, S. Bilguun, and S. Ren, “Operation Analysis of Fast Charging Stations With Energy Demand Control of Electric Vehicles.” IEEE Trans. Smart Grid , vol.6, no.4, pp.1819 – 1826, July 2015. S. Cheng, Y. C. Hsu, C. H. Gan, and Y. B. Lin, “Charge scheduling of electric vehicles in highways.” Mathematical and Computer Modelling, vol.57, no.11, pp.2873-2882, June 2013. W. Tushar, W. Saad, H. V. Poor, and D. B. Smith, “Economics of electric vehicle charging: A game theoretic approach.” IEEE Trans. Smart Grid, vol.3, no.4, pp. 1767-1778, Dec. 2012.

[13]

[14]

[15]

[16] [17]

[18]

[19] [20] [21] [22]

[23]

[24]

[25]

[26] [27]

[28]

[29]

1424

S. Bahrami, M. Parniani, “Game theoretic based charging strategy for plug-in hybrid electric vehicles.” IEEE Trans. Smart Grid, vol.5, no.5, pp. 2368-2375, Sept. 2014. W. Lee, X. Lin, R. Schober, and V.W.S. Wong, “Electric Vehicle Charging Stations With Renewable Power Generators: A Game Theoretical Analysis.” IEEE Trans. Smart Grid, vol.6, no.2, pp. 608617, March 2015. J.M. Foster, G. Trevino, M. Kuss, and M.C. Caramanis, “Plug-in electric vehicle and voltage support for distributed solar: theory and application.” IEEE Systems Journal, vol.7, no.4, pp. 881-888, Dec. 2013. E. Erik, M. Milligan, and B. Kirby. “Operating reserves and variable generation.” National Renewable Energy Laboratory, CO, Rep.TP5500-51978, August 2011. M. Kintner-Meyer, “Regulatory policy and markets for energy storage in North America.” Proceedings of the IEEE, vol.102, no.7, pp.1065-1072, July 2014. J. P. Torreglosa, P. García-Triviño, L. M. Fernández-Ramirez, and F.Jurado, “Decentralized energy management strategy based on predictive controllers for a medium voltage direct current photovoltaic electric vehicle charging station.” Energy Conversion and Management, vol.108, pp.1-13, Jan. 2016. Q. Wang, C. Zhang, Y. Ding, G. Xydis, J. Wang, and J. Østergaard, “Review of real-time electricity markets for integrating Distributed Energy Resources and Demand Response.” Applied Energy, vol.138, pp. 695-706, Jan. 2015. Balakrishnan, A. V., and M. Thoma. "Stackelberg Differential Games in Economic Models." Berlin: Springer2Verlag, 1984. Mas-Colell, Andreu, Michael Dennis Whinston, and Jerry R. Green. Microeconomic theory. Vol. 1. New York: Oxford university press, 1995. Olivares, Daniel E., Claudio A. Cañizares, and Mehrdad Kazerani. "A centralized energy management system for isolated microgrids." Smart Grid, IEEE Transactions on IEEE Trans. Smart Grid, vol 5, no. 4, pp. 1864-1875, 2014 Basar, Tamer, and Geert Jan Olsder. Dynamic noncooperative game theory. Vol. 23. Siam, 1999. Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004. Lai, Young-Jou. "Hierarchical optimization: a satisfactory solution." Fuzzy Sets and Systems, vol. 77, no. 3, pp. 321-335, 1996. De Wolf, Daniel, and Yves Smeers. "A stochastic version of a Stackelberg-Nash-Cournot equilibrium model." Management Science, vol. 43, no. 2, pp. 190-197, 1997. S. Siddiqui and S. A. Gabriel, “An SOS1-based approach for solving MPECs with a natural gas market application,” Netw. Spat. Econ., vol. 13, no. 2, pp. 205–227, Jun. 2013. C. Bliek1ú, B. Pierre, and L. Andrea, “Solving mixed-Integer quadratic programming problems with IBM-CPLEX: a progress report.” Proceedings of the Twenty-Sixth RAMP Symposium, Tokyo, Japan, Oct. 2014. Electric Power Monthly[Online]. Available: https://www.eia.gov/electricity/monthly/epm_table_grapher.cfm?t=epmt _5_6_a Tesla Supercharger [Online]. Available: https://www.teslamotors.com/supercharger?redirect=no. H. Quan, D. Srinivasan, and A. Khosravi, “Short-term load and wind power forecasting using neural network-based prediction intervals.” IEEE Trans. Neural Networks and Learning Systems, vol.25, no.2, pp.303-315, Feb. 2014. Pricing Data. [Online]. Available: www.nyiso.com/public/markets_operations/market_data/pricing_data/in dex.jsp Solar Power Data for Integration Studies Datasets[Online].Available:www.nrel.gov/electricity/transmission/solar_ integration_dataset.html