Adaptive Control for an HVDC Transmission Link with FACTS and a Wind Farm

Yufei Tang, Haibo He, Senior Member, IEEE, and Jinyu Wen, Member, IEEE

AbstractDue to the nonlinearity, uncertainty and complexity of the power system, it is a challenging task to design an effective control approach based on the exact model using traditional methods. In this paper, we investigate the application of a novel approximate dynamic programming (ADP) architecture, goal representation heuristic dynamic programming (GrHDP), to a large benchmark power system. Unlike traditional ADP design with an action network and a critic network, GrHDP integrates the third network, a goal network, into the actorcritic design (ACD) to automatically and adaptively build an internal reinforcement signal representation to facilitate learning and optimization. Then the GrHDP is employed to control the benchmark power system including a DFIG based wind farm and a STATCOM with HVDC transmission. Various power system states, including the voltage of STATCOM, current of DFIG and DC current of HVDC inverter, are provided to the GrHDP controller to generate three adaptive supplementary control signals. These adaptive supplementary control signals are then provided to the STATCOM controller, DFIG rotor side controller and HVDC master control, respectively. This control structure is validated in Matlab/Simulink to demonstrate its effectiveness in power system control. Index TermsPower system stability control, smart grid control, adaptive dynamic control (ADP), wind farm, high voltage direct current (HVDC) transmission.

HVDC transmission plays an important role in the smart grid for renewable power generation and AC grid interconnection. For instance, HVDC control to improve power system stability has been investigated in recent years [3][6]. In [3], design of multiple HVDC damping control based on wide area monitoring system (WARMS) has been discussed. The relative gain array (RGA) analysis is employed to nd the suitable input-output pairing for the multiple HVDC damping controller. A real power system with 800kV HVDC transmission in China has been investigated in [5]. Suggestions of reinforcing demand side management are proposed to solve the problem of under voltage load tripping. Coordinated control between DFIG and HVDC has been studied in many papers [7][10]. The DFIG and HVDC system modeling, control, and engineering studies are carried out in [9] and [10]. During the past decades, computational intelligence (CI) based optimal control on power system has shown very promising results [11]. Extensive work have been done on optimal neuro-control based on ACD to improve power system dynamics and transient performance [12][14]. In [14], the authors proposed a heuristic dynamic programming (HDP) based coordinated reactive power control of a large wind farm with STATCOM. This HDP controller could improve the performance of the DFIG in the power grid under grid fault. However, this HDP controller requires a model neural network which needs to be sufciently pre-trained off-line. The training quality of the model constraints the control performance. A multi-infeed HVDC power system was investigated in [13] to test the control effect of direct HDP (model free) in inter-area oscillation damping. It shows that HDP is a powerful method to damp inter-area oscillation and to increase power system stability. In this paper, a new ADP structure, goal representation heuristic dynamic programming (GrHDP) [15][17], is employed to control an HVDC transmission link with STATCOM and a DFIG based wind farm. The rest of the paper is organized as follows: In Section II the system model is introduced. Section III introduces the GrHDP controller design for the benchmark power system. Then in Section IV simulation results in Matlab/Simulink are provided. Section V draws some conclusions and discussions.

I. I NTRODUCTION

HE goal of a future smart grid requires an intelligent integration of transmission and distribution with power generation [1]. Such a smart grid will have some key characteristics, such as improved reliability (especially with the new complexity challenges of high voltage bulk power transmission and cyber security issues), increased efciency of generating, transmission and distribution, accommodating the growth of renewable energy sources like solar and wind, and enabling electric vehicles integration. To achieve this goal, new technologies (e.g., energy storage, advanced power electronics, etc) and new control methods (e.g., adaptive control) are needed. In this paper, we focus on adaptive dynamic control for HVDC transmission link with a STATCOM and a DFIG based wind farm.

This work was supported in part by the National Science Foundation (NSF) under grant ECCS 1053717 and CNS 1117314, and the National Natural Science Foundation of China (NSFC) under grant 51228701. Haibo He and Yufei Tang are with the Department of Electrical, Computer and Biomedical Engineering, University of Rhode Island, Kingston, RI 02881, USA (email: {he, ytang}@ele.uri.edu). Jinyu Wen is with the College of Electrical, Electronic and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China. (email: jinyu.wen@mail.hust.edu.cn)

978-1-4673-4896-6/13/$31.00 2013 IEEE

II. S YSTEM MODEL FOR THE CONTROLLER DESIGN A. Modeling the overall system The benchmark power system considered in this paper has two areas interconnected by HVDC transmission (Fig. 1). It comprises a 1000M W synchronous generator, a 600M W wind farm and a 1000M W thyristor-based HVDC [7]. Power from the synchronous generator and the wind farm is transmitted to the 50Hz grid through the HVDC connection. The HVDC is a 500kV 2kA dc link. Converters of HVDC are connected through a 300km transmission line. Reactive power required by the converters in HVDC is provided by a set of capacitor banks plus 11th, 13th and high pass lters for a total of 600M var on each side, whereas, a STATCOM is connected to Bus3, together with the large wind farm, to provides ne reactive power control to improve the system dynamic under fault condition [8].

Fig. 2.

Schematic diagram of DFIG control

C. Modeling the STATCOM control The STATCOM and its controller are shown in Fig. 3. It is a shunt device of the Flexible AC Transmission System (FACTS) family using power electronics to control power ow and improve transient stability on power grid [9] [10]. The STATCOM regulates voltage at its terminal by controlling the amount reactive power injected into or absorbed from the power grid which depends on the system voltage. In this paper, the STATCOM is operated with constant reactive power mode. A constant Qref divided by the measured voltage to form the reactive current reference Iqref used by the current regulator block. The output of the current regulator block is the angle which is the phase shift. Then ring pulses generator generates pulses for the four inverters from the output of the phase locked loop (PLL) and the current regulator output ( angle).

Fig. 1.

The overall power system conguration

B. Modeling the DFIG based wind farm control In this paper, the wind farm will be represented by one WT with DFIG system for system level control. In practice, the DFIGs in a wind farm are always located within a relatively small area where the wind speed could be assumed the same. From a practice point of view, all of the DFIGs should be controlled independently without mutual interaction. It might be useful in some cases that through control the stator ux linkage reference in some wind generators to provide more reactive power during a specic period of time [8]. The design for DFIG control takes into consideration the maximum wind power extraction and reactive power compensation as showed in Fig. 2. The control mechanism is implemented through a traditional nested loop structure which has an outer speed/active power and reactive power control loop and an inner d-q rotor current control loop [7]. Rotor d-q reference currents can be effectively estimated by direct computational approach from the real and reactive power references which eliminates the power loop controller of the nested control structure. In our previous research [14], the supplementary control signal Qsup is added at the outer control loop. To achieve faster dynamic control, the supplementary control signal is added in the inner d-q rotor current loop controller in our current research.

Fig. 3.

Schematic diagram of STATCOM control

D. Modeling the HVDC transmission control A typical HVDC supplementary control scheme [18] is shown in Fig. 4. The active power order is divided by the measured DC voltage to form the DC current order Iorder . Then this reference value is sent to the Voltage Dependent Current Order Limiter (VDCOL) at the rectier side and inverter side, respectively. The function of VDCOL

is automatically reduces the reference current set point when DC line voltage decreases (e.g. during a DC line fault or a severe AC fault). Reducing the reference currents also reduce the reactive power demand on the AC system, helping the system to recover from the fault. After VDCOL, PI controller is employed to form the ring angle R of the rectier and i of the inverter. Both the current order on the rectier side and the voltage order on the inverter side could be modulated. Also, the supplementary control signal could be added to the ring angle directly. In this paper, the supplementary control signal is added before the VDCOL on the rectier side.

to generate three supplementary control signals, IdR sup , Idr sup and QS sup . The objective function or reinforcement signal of the GrHDP controller is as follows: (t) + 0.5Id2 (t 1) + 0.1Id2 (t 2) r(t) = [Id2 R R R 2 2 2 (t 2) +Iwind (t) + 0.5Iwind (t 1) + 0.1Iwind 2 2 2 +VS (t) + 0.5VS (t 1) + 0.1VS (t 2)] (2) As we have discussed above, three supplementary control signals are generated by the GrHDP controller and then sent to the DFIG rotor current controller, STATCOM reactive power regulator, and HVDC master control, respectively. The control principle is decried as follows. When the system is in steady state, the GrHDP three input signals, IdR , Iwind and VS are zero. Then the GrHDP controller three output supplementary control signals, IdR sup , Idr sup and QS sup will be zero. When there is a grid fault, the voltage of the STATCOM and DFIG will fall to a low level. This DFIG voltage sag can cause an imbalance between the turbine input power and the generator output power, which will lead a high current in the stator windings. Meanwhile, the power transfer on HVDC will be affected by the voltage sag, and therefore, a sag of DC current and DC voltage. The GrHDP output signals will be varying depending on the input signals according to equation (2) (the utility function). The STATCOM and DFIG output reactive power will satisfy the system requirements, say compensate enough reactive power to improve voltage sag during the fault and damp the system oscillation after the fault. Also, the master control of HVDC will change the reference current sent to the rectier to damp the system oscillation. IV. C ASE STUDIES Simulations are performed in Matlab/Simulink to verify the improvement in stability by the proposed GrHDP controller. DFIG side single-phase-ground-fault and inverter side three-phase-ground-fault are used in the simulations. A. Case one: DFIG side single-phase-ground-fault

Fig. 4.

Schematic diagram of HVDC control

III. G R HDP CONTROLLER DESIGN A. Goal representation heuristic dynamic programming The GrHDP controller includes three main parts: an action network, a critic network and a reference network [15]. The action network produces control signal u(t) according to a learning policy represented by the approximating network, while the reference network to provide the internal reinforcement signal (internal goal representation) s(t), to interact with the operation of the critic network to approximate the cost and reward function J by minimizing the Bellman function as follows: J (x(t)) = min{U (x(t), u(t)) + J (x(t + 1))}
u(t)

(1)

DC current of the rectifier (p.u)

where x(t) is the state vector of the system, U is the utility function, and is a discount factor. These three parts are usually implemented by neural networks because of their universal approximation capability and the associated backpropagation learning algorithm. The readers could refer to [15] [16] [17] for the back propagation rules and weights setting. B. GrHDP based system controller The aim of the control system is to regulate the active power and reactive power of the power grid while regulating the power ow via control of the HVDC link and the reactive power via control of the STATCOM and DFIG. The detail of the proposed control structure is shown in Fig. 5. Three feedback signals, IdR (deviation of the HVDC rectier current), Iwind (deviation of the wind farm current) and VS (deviation of the STATCOM voltage), are employed

Without GrHDP With GrHDP

1.5

0.5

2.1

2.2

2.3

2.4

2.5 Time (s)

2.6

2.7

2.8

2.9

Fig. 6.

DC current of HVDC rectier in case one

A single phase ground fault is applied at the DFIG side which lasts for 0.1s. Fig. 6 to Fig. 9 shows the system

Fig. 5.

The proposed control structure

dynamics with and without the GrHDP supplementary controller, respectively. Specically, Fig. 6 shows the DC current of HVDC rectier, Fig. 7 shows the active power of DFIG, Fig. 8 shows the reactive power of DFIG , and Fig. 9 shows the reactive power of STATCOM. It can be observed that, with the proposed control method, the system has much better performance.

800

Without GrHDP With GrHDP

600
Reactive power of wind farm (MVar)

400

200

200

400 700

Without GrHDP With GrHDP

2.1

2.2

2.3

2.4

2.5 Time (s)

2.6

2.7

2.8

2.9

600

Active power of wind farm (MW)

500

Fig. 8.

Reactive power of DFIG in case one

400

300

200

B. Case two: Inverter side three-phase-ground-fault

2 2.1 2.2 2.3 2.4 2.5 Time (s) 2.6 2.7 2.8 2.9 3

100

Fig. 7.

Active power of DFIG in case one

Fig. 10 shows the parameters of the GrHDP controller. Here W c, W a and W r are the weights of the critic network, action network and reference network, respectively. J is the reward function. Reinforcement signal is the utility function as mentioned in equation (2). Sr is the internal goal provided by the reference network. It can be observed that the weights are converged after the fault. The controller is trying its best to bring the system back to its steady state.

A three-phase ground fault is applied at the inverter side which lasts for 0.1s. Fig. 11 to Fig. 14 shows the system dynamics with and without the GrHDP supplementary controller, respectively. Specically, Fig. 11 shows the DC current of HVDC rectier, Fig. 12 shows the active power of DFIG, Fig. 13 shows the reactive power of DFIG , and Fig. 14 shows the reactive power of STATCOM. It can be observed that, with the proposed GrHDP controller, the fault recovery ability of the system has been improved. The results in case two are in consistent with those in case one, which means the proposed method is robust under different fault conditions.

Wc
0.2 0.5 0.1 0 0.1 0.2 0

Wa

0.5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 x 10 2

4

0.2

0.4

0.6

0.8

1.2

1.4

1.6

1.8 x 10

2
4

Wr
1 0.5 0 0.5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 x 10 2
4

J
0 2 4 6 8 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 x 10 2
4

Reinforcment
0 2 4 6 8 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 x 10 2
4

Sr
1 0.8 0.6 0.4 0.2 0 0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 x 10 2
4

Fig. 10.

The parameters of GrHDP in case one

300

Without GrHDP With GrHDP

2 1.8 1.6
DC current of the rectifier (p.u)

Without GrHDP With GrHDP

200
Reactive power of STATCOM (MVar)

100

1.4 1.2 1 0.8 0.6 0.4

100

200

300

0.2 0 2 2.1 2.2 2.3 2.4 2.5 Time (s) 2.6 2.7 2.8 2.9 3

2.1

2.2

2.3

2.4

2.5 Time (s)

2.6

2.7

2.8

2.9

Fig. 9.

Reactive power of STATCOM in case one

Fig. 11.

DC current of HVDC rectier in case two

V. C ONCLUSIONS A coordinate control strategy for HVDC transmission with FACTS and a wind farm is presented in this paper. The proposed strategy, based on a new GrHDP approach, achieves good performance on the benchmark power system under both DFIG side single-phase-ground-fault and inverter side three-phase-ground-fault, enhancing the system ride through capability and damping the system oscillation. Furthermore, the proposed GrHDP control yields a robust performance under different system conditions. R EFERENCES
[1] U.S. Dept. of Energy, [Online]. Available: http://energy.gov/smart-grid.
600

Without GrHDP With GrHDP

Active power of wind farm (MW)

500

400

300

200

100 2 2.1 2.2 2.3 2.4 2.5 Time (s) 2.6 2.7 2.8 2.9 3

Fig. 12.

Active power of DFIG in case two

500 400 300 200 100 0 100 200 300 400 500 600 2 2.1 2.2 2.3 2.4 2.5 Time (s) 2.6 2.7 2.8 2.9 3

Without GrHDP With GrHDP

Fig. 13.

Reactive power of DFIG in case two

[13] C. Lu, J. Si, and X. R. Xie, Direct Heuristic Dynamic Programming for Damping Oscillations in a Large Power System, IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 1008-1013 , 2008. [14] Y. Tang, H. He, Z. Ni, J. Wen, and X. Sui, Reactive Power Control of Grid-Connected Wind Farm Based on Adaptive Dynamic Programming, Neurocomputing, 2012 (in press). [15] H, He, Z. Ni, and J. Fu, A three-network architecture for on-line learning and optimization based on adaptive dynamic programming, Neurocomputing, vol. 78, issue 1, pp. 3-13, 2012. [16] Z. Ni, H. He, D. Zhao, and D. V. Prokhorov, Reinforcement Learning Control Based on Multi-Goal Representation Using Hierarchical Heuristic Dynamic Programming, in Proc. IEEE World Congress on Computational Intelligence, Brisbane, Australia, 2012. [17] H. He, Self-Adaptive Systems for Machine Intelligence, Wiley, 2011. [18] P. Kundur, Power System Stability and Control., New York: McGraw-Hill, 1994.

Reactive power of wind farm (MVar)

300

Without GrHDP With GrHDP

Reactive power of STATCOM (MVar)

200

100

100

200 2 2.1 2.2 2.3 2.4 2.5 Time (s) 2.6 2.7 2.8 2.9 3

Fig. 14.

Reactive power of STATCOM in case two

[2] IEEE SmartGrid, [Online]. Available: http://smartgrid.ieee.org. [3] J. Wang, C. Fu, and Y. Zhang, Design of WAMS-Based Multiple HVDC Damping Control System, IEEE Trans. Smart Grid, vol. 2, no. 2, pp. 363-374, 2011. [4] A.E. Leon, J.M. Mauricio, J.A. Solsona, and A. Go andmez-Expo andsito, Adaptive Control Strategy for VSC-Based Systems Under Unbalanced Network Conditions, IEEE Trans. Smart Grid, vol. 1, no. 3, pp. 311-319, 2010. [5] Y. Yang, X. Yang, F. Qian, L. Li, G. Wu, S. Lin, and M. Liu, Effect of 800 kV Yun-Guang ultra HVDC transmission system on voltage prole in Guangdong Power Grid and corresponding enhancement measures, Innovative Smart Grid Technologies Conference Europe (ISGT Europe), 2010. [6] W. Kuehn, Real-time method to prevent voltage collapse and power instability of HVDC systems, Innovative Smart Grid Technologies Conference Europe (ISGT Europe), 2010. [7] H. Yin, L. Fan, and Z. Miao, Fast Power Routing Through HVDC, IEEE Trans. Power Del., vol. 27, no. 3, pp. 1432-1441, 2012. [8] D. Xiang, L. Ran, J.R. Bumby, P.J. and Tavner, and S. Yang, Coordinated control of an HVDC link and doubly fed induction generators in a large offshore wind farm, IEEE Trans. Power Del., vol. 21, no. 1, pp. 463-471, 2006. [9] S.V. Bozhko, R. Blasco-Gimenez, R. Li, J.C. Clare, and G.M. Asher, Control of Offshore DFIG-Based Wind Farm Grid With LineCommutated HVDC Connection, IEEE Trans. on Energy Convers., vol. 22, no. 1, pp. 71-78, 2007. [10] S.V. Bozhko, G.M. Asher, R. Li, J.C. Clare, and L. Yao, Large Offshore DFIG-Based Wind Farm With Line-Commutated HVDC Connection to the Main Grid: Engineering Studies, IEEE Trans. on Energy Convers., vol. 23, no. 1, pp. 119-127, 2008. [11] R. Harley, and J. Liang, Computational Intellifence in Smart Grid, IEEE Symposium Series on Computational Intelligence (SSCI), 2011. [12] W. Qiao, R. G. Harley, and G. K. Venayagamoorthy, Coordinated Reactive Power Control of a Large Wind Farm and a STATCOM Using Heuristic Dynamic Programming, IEEE Trans. on Energy Convers., vol. 24, no. 2, pp. 493-503, 2009.