IEEE - IECON'2000, October 22-28, Nagoya, Japan, 2000
CONSTRAINED NEURAL MODEL PREDICTIVE CONTROL WITH GUARANTEED FREE OFFSET P. Gil†‡
J. Henriques†
A. Dourado†
H. Duarte-Ramos‡
[email protected]
[email protected]
[email protected]
[email protected]
‡
†
Electrical Engineering Department, UNL CISUC-Informatics Engineering Department, UC Pólo II, Pinhal de Marrocos, 3030 Coimbra 2825-114 Monte de Caparica Phone: +351 239790000, Fax: +351 239701266 Phone: +351 212948545, Fax: +351 212948532 Portugal Portugal
Abstract An extended model-based predictive control scheme is proposed and implemented on a bench three-tanks system. This structure is based on a constrained local instantaneous linear model-based predictive controller complemented with a static offset compensator for guaranteeing that tracking errors converge to zero in a finite time. A non-linear state-space neural network architecture trained offline is used for modelling purposes and forming a seed from where linear models are extracted at each sampling time. Results from experiments show that this extended MPC scheme ensures a good tracking performance with zero steady-state offsets, in spite of modelling errors. Index Terms: non-linear systems; model predictive control; constrained optimisation; recurrent neural networks.
1. Introduction For the last two decades or so, the control community has actively been witnessing an increasing acceptance of model-based predictive control (MPC) methodologies as a valuable mean for solving practical control problems, particularly in the process industries. This perceptible success of MPC strategies can be attributed to several factors involving its inherent ability to handle input and output constraints, time delays and the incorporation of an explicit model of the plant into the optimisation problem [1]. The MPC history can clearly be traced back to the late 1970’s with the Model Algorithm Control and the Dynamic Matrix Control techniques [2]. In both schemes a
linear description of the plant is used. More recently, and pushed by the non-linear nature of most industrial processes, some research efforts have been placed in the development of reliable non-linear model predictive control strategies (NMPC). In view of these methodologies, two fundamental research directions can be found in the literature: implementation of NMPC schemes based on explicit non-linear models either based on first-principles (white-box), e.g. [4], or black-box modelling, e.g. [5]; non-linear receding horizon control law approximation making use of general function approximators, such as neural networks, and construction techniques, given a training data stemming from a finite horizon optimisation problem, considering a sufficiently large set of admissible initial states, e.g. [3]. The main drawback of the on-line NMPC approaches is related to the solution of a non-convex optimisation problem, since the convergence to a feasible/optimal solution and stability cannot be guaranteed in advanced, not to mention the unpredictable computation time that is required, while in the receding horizon control law case derived from “identification” extrapolations are not reliable at all. Another issue concerning non-linear control is that for many systems it is difficult (or even impossible) and expensive to come up with an accurate physical model, required by MPC techniques, mainly due to the complexity of the underlying phenomena or the lack of some specific critical parameters. In these circumstances neural networks have proved to work quite well in the identification of non-linear systems based on input-output data (see e.g. [6]). Though neural networks are universal approximators [7], [8], they are quite dependent on the quality of the data set. This feature in conjunction with a bounded number of iterations in the learning stage leads inevitably to a model mismatch, which ultimately is responsible for a steady state error. In [9] it is suggested
adding up to the system’s step response an estimate of modelling errors. Alternatively, the incorporation in the control loop of an offset compensator might be considered. In view of the issues above, this paper presents an implementation on a bench three-tanks system of a constrained local instantaneous linear model predictive control (LIMPC) scheme. The explicit linear model is extracted at each sampling time from the state-space neural network model of the plant, previously trained offline. In order to get rid of static offsets the system’s outputs are externally fed back such that exponentially weighted errors are used in an integral way to change appropriately the reference signals. Given the structure of the compensation element only small errors are truly taken into account by the integrator and thus windup occurrences are not expected.
actions, the open-loop optimisation problem can be stated in the following way: 2 P min J = min y( k + i | k ) − r ( k + i ) Q u u i=1 M−1 P− 1 (2) 2 2 + ∆ u( k + i | k ) S u( k + i | k ) R + i= 0 i= 0
∑
∑
Model-based predictive control is a discrete-time technique where an explicit dynamic model of the plant is used to predict the system’s outputs over the finite prediction horizon P when control actions are manipulated throughout the finite control horizon M . At time step k the optimiser computes on-line the optimal open-loop sequence of control actions such that the predicted outputs follow a pre-specified reference and taking into account possible hard and soft constraints. Only current control actions u ( k | k ) are actually
implemented on the plant over the time interval [k ,k + 1) .
Next, at time step k + 1 , the prediction and control horizons are shifted ahead by one step and a new optimisation problem is solved taking into account the most recent measurements, and the control action fed to the plant in the previous time interval, u ( k | k ) . Let the linearised discrete-time dynamics of a general non-linear system be described in the state-space form as follows: x( k + 1) = Φ x( k ) + Γ u ( k ) + η (1) y ( k ) = Ξ x( k ) where Φ ∈ ℜ n ×n , Γ ∈ ℜ n ×m and Ξ ∈ ℜ p×n are, respectively, the state, input and output matrices; x ∈ ℜ n is the state vector, u ∈ ℜ m is the control vector and y ∈ ℜ p the output vector; η ∈ ℜ n is a constant vector related to the first term of the Taylor expansion, which is zero at any equilibrium point. Considering a 2-norm for the cost functional and linear constraints on the inputs and outputs of the system and additionally bounds on the rate of change of control
∑
i
i
subject to system dynamics (1) and to the following inequalities:
y min ≤ y ( k + i k ) ≤ y max , i = 1,K , P, k ≥ 0
u min ≤ u ( k + i k ) ≤ u max , i = 0, K, P − 1, k ≥ 0 ∆ u ( k + i k ) ≤ ∆ u max , i = 0,K , M − 1, k ≥ 0
(3)
∆ u ( k + i k ) = 0, i = M ,K , P − 1, k ≥ 0 p× p
with Qi ∈ ℜ
2. Extended MPC Formulation
i
, Ri ∈ ℜ
m ×m
, Si ∈ ℜ
m ×m
, ∆u ∈ ℜ
m
is
p
the control increment vector and r ∈ ℜ is the reference signal. Given the convexity of the optimisation problem above (quadratic objective function and linear constraints) any particular solution is a global optimum and, in addition, the open-loop optimal control problem can be restated as a quadratic programming problem (4) and (5). minimise subject to
1 J( ∆ u~ ) = h T ∆ u~ + ∆ u~ T H ∆ u~ 2 T ~ A ∆u ≤ b
(4) (5)
where A ∈ ℜ m M × ( 4m M + 2 pP ) , b ∈ ℜ ( 4m M + 2 pP ) and ∆ u~ ∈ ℜ m M is the extended control increments over the control horizon. The cost function’s gradient h ∈ ℜ and the Hessian H ∈ ℜ
P − 1 hlT = 2 x oT ( ΞΦ i = l − 1
∑
+ [Γ u o + η ]
T
m M ×m M
are given by:
ΞΦ q Γ q= 0
i− l+ 1
)Q ∑
i+ 1 T
i+ 1
ΞΦ s Γ s= 0
∑ ∑ (ΞΦ ) Q ∑ P− 1
i
i− l+ 1
q T
i+ 1
i= l− 1
q= 0
i− l+ 1 T − ( r ( i + 1)) Qi + 1 ΞΦ q Γ q= 0 i = l − 1 P− 1 + u oT Ri ; {l = 1, K M } i= l− 1 P− 1
∑
mM
(6)
∑
∑
i P − l i T H ll = 2 Γ T ∑ ∑ ( ΞΦ q ) Qi + 1 ∑ ΞΦ q Γ i = 0 q = 0 q= 0 P− 1 + ∑ Ri + S l − 1 ; {l = 1,K, M } i= l− 1
(7)
H
l p
i − p+ 1 P − 1 i − l+ 1 T = 2 Γ T ΞΦ q Qi + 1 ΞΦ i = l− 1 q = 0 q=0 P− 1 } + R i ; {p = 1,K, M ; p ≠ l i = l− 1
∑
∑ (
)
∑
q
Γ
3. Neural Network Modelling (8)
∑
In order to prevent from the effect of modelling errors that are responsible for static offsets, we propose here the inclusion of an offset compensator, as depicted schematically in Fig. 1. This device works partially as an integrator where the control error is previously weighted before being added up to the cumulative error. Next, this summation is added up to the true reference signal before being supplied to the MPC structure. r
Com pe n s ator
r~
u
M PC
xˆ
Pl an t
y
O b s e rve r
Fig. 1. Controller Structure. Since the weighting factor is given by a decreasing exponential according to (9), only small errors are indeed taken into account. This feature enables that merely in the vicinity of the set point the reference signal is effectively manipulated, avoiding this way undesirables windup effects over the reference signal. Additionally, the compensator should be able to reset the cumulative error whenever a changing in the true reference takes place, in order to prevent the MPC component from receiving a miscalculated set point, which could ultimately have a disastrous consequence on the system’s behaviour, such as an excess of overshoot for the system’s output. This task is here accomplished by multiplying the previous cumulative error ϑ ( k − 1) by a Dirac delta function having as argument the difference between the present and next set points. The above issues forming the offset compensator paradigms, i.e. the reset of the cumulative error and the insensitivity to significant errors, can be stated as follows:
(
ϑ ( k ) = ϑ ( k − 1)δ(r ( k + 1) − r ( k )) + e ( k )e xp − α e ( k ) r~ ( k + i ) = r ( k + i ) + ϑ ( k ), i = 1,K, P
)
+
where f : ℜ n × ℜ m → ℜ n and h : ℜ n × ℜ m → ℜ p . For capturing the system’s dynamics above we resort to an affine deterministic non-linear black-box model based on the recurrent neural network architecture depicted schematically in Fig. 2.
u (k )
q -1
WH
WA ϕ
WD
ξ ( k + 1)
WC
yˆ( k + 1)
WB WF
q -1
Fig. 2. Recurrent neural network structure. (9)
where δ is the Dirac delta function, e ( k ) is the current tracking error and α ∈ ℜ exponential function.
Black-box models based on neural networks have become in the last few years a reliable way for describing non-linear systems dynamics, given input-output measurements. Among various architectures, state-space neural models (recurrent neural networks) have proved to be potentially more efficient and less demanding in terms of the number of parameters than feedforward (inputoutput) counterparts [10]. Basically, a neural network consists of a number of cell units, also known as processing elements or neurons, which are connected to each other by means of an activation function either linear or non-linear. A numerically adjusted weight in the learning stage quantifies the synaptic strength between neurons. Neural networks can be arranged topologically in a feedforward or recurrent way. In a feedforward network synaptic signals flow via unidirectional connections between consecutive layers, while in the recurrent case it is allowed feedback loops in a number of neurons. For a comprehensive review on neural networks the reader is referred to [12]. Consider the general deterministic non-linear system (10) to be identified on the basis of observed data, which are collected by carrying out an identification experiment. x&( t) = f( x ( t),u ( t)), x ( t0 ) = x0 (10) y ( t) = h ( x ( t),u ( t))
\ 0 is the coefficient of the
In this neural network architecture ξ ( k ) ∈ ℜ
current neural state vector, yˆ( k ) ∈ ℜ
p
n
is the
is the predicted
output vector, W A ,W B ,W C ,W D ,W F and W H are weight matrices of appropriate dimensions representing the
synaptic strengths between neural units, ϕ is a non-linear vector activation function, typically hyperbolic tangent or sigmoid and q − 1 is the backward shift operator. The choice for this particular topology comprising three layers was made under the premise that the selected structure should belong to the recurrent neural network kernel and given the weak non-linear nature dynamics of the plant to be modelled in this paper, the network should include linear activation functions as well. This latter assumption was taken since hybrid networks are known to perform better than those having only non-linear activation functions when non-linear dynamics are not very significant, [13]. Another issue concerning the network’s structure is that it should be straightforward to derive a corresponding state-space form in order to enable its incorporation within model-based predictive control framework. Having this in mind, the model resulting from the training of the adopted topology should be restated first in state space form. When writing out the recurrent neural network equations considering a hyperbolic tangent mapping as the activation function, it is clear that by substituting the predicted outputs for yˆ( k ) = Cξ ( k ) the state space form given by (11) is obtained.
ξ ( k + 1) = D tan h ( E ξ ( k )) + F ξ ( k ) + G u ( k ) yˆ( k ) = C ξ ( k ) where
G∈ ℜ
matrices n ×m
C∈ ℜ
p× n
,
D ,E , F ∈ ℜ
are evaluated according to (12). C = W C D = W D E = W A + W FW C F = W H G = W B
n ×n
∑
N k =1
4.1
Process Description
The bench three-tanks system depicted schematically in Fig. 3, consists of three plexiglas cylinders supplied with distilled water. One of the tanks ( T3 ) is connected to the other two tanks by circular cross section pipes provided with manually adjustable ball valves. In the tank T2 is located the main outlet of the system that is connected to the collecting reservoir by means of a circular cross section nozzle and an outflow ball valve. Additionally, at each tank lies another connection to the reservoir, enabling the injection of disturbances under the form of leakages. Two diaphragm pumps are available for pumping liquid from the bottom reservoir to T1 and T2 tanks. Pum p 1
Pum p 2 T1
h1
(11)
T3
h3
T2
h2
and
Fig. 3. The three-tanks system schematic. (12)
In this paper the estimation of the networks weights is carried out offline (batch training) by minimizing a sum of squared prediction errors J(W ) =
4 Case Study: The Three-Tanks System
[y(k ) − yˆ(k W )]
2
through the Levenberg-Marquardt algorithm. Weights are updated iteratively according to: −1 (13) ∆ W = − ( H + λI) ∇ J(W )
where H denotes the Hessian of the cost-function J(W ) ,
∇ J(W ) is the gradient, λ ∈ ℜ + and I is an identity matrix of appropriate dimensions. The Levenberg-Marquardt algorithm has a very attractive feature as it spans from a steepest descent type method for large values of λ , to a Gauss-Newton’s method, when λ → 0 . A straightforward strategy to update the parameter λ can be found in [11].
4.2.
Identification and Control
For the identification purposes open-loop experiments were carried out on the bench process aiming to collect informative enough data to be used in the model estimation stage. These experiments consisted in the system’s excitation with step and pseudo random binary signals, choosing a sampling interval of 1 second. Two records were selected from the experiments: one data set was used for training the neural network chosen (Fig. 2), while the other one was reserved to cross-validation tests. After several model estimation/validation trials on the basis of a number of different neurons in the hidden, a suitable neural predictor was obtained. This neural network involves two neurons for the input and output layers and three neurons in the hidden layer. In Fig. 4 are compared the model outputs and the sampled outputs for the validation data set. As can be observed the neural model is able to predict fairly well the system’s dynamics.
0.6
remove completely steady state offsets. These deviations are attributed mainly to modelling errors with a lesser contribution from the first order Taylor expansion.
0.4
0.5
0.3
0.4
0.2
M e asure d
0.1
Levelm[]
Levelm[]
0.5
M ode l
0 200
300 400 Tim e [s]
500
600
4.0
6.0
3.5
5.5 u1
3.0
4.5 4.0
u2
1.5
3.5
1.0
3.0
0.5
2.5
0
0
100
200
300 400 Tim e [s ]
0
500
2.0 600
Fig. 4. Validation data set. The MPC scheme implementation on the bench process is based on a local instantaneous linear model extracted from the discrete-time neural predictor at each sampling time. This linear model is then used in the openloop constrained optimal control problem choosing the prediction horizon as P = 3 time steps and control horizon as M = 1 . Given the presence of physical bounds on flow rates and levels the following inequality constraints were taken into account in the optimisation problem (4) and (5):
[0 0 0]T ≤ x ≤ [0.6 [0 0]T ≤ u ≤ [5.85
r2
0.2
5.0
2.5 2.0
0.3
h2
0.1
0.6 0.6 ] m T
6.16 ] l/min
(14)
T
In addition to these constraints upper and lower bounds in control actions increments have to be considered as well in order to influence the rate of change associated to those variables. In the experiments these constraints were chosen as ∆ u ≤ [0.9 6 0.9 6 ] l/min. T
To test the performance of the linear instantaneous neural model predictive control strategy a free-disturbance tracking experiment was conducted, being the results displayed in Fig. 5. As can be observed the neural modelbased predictive controller though guaranteeing a stable response without any constraint violation it is not able to
0
100
200
300
400
500
600
Tim e [s ]
(a) Set-points and outputs. 8 Flow Rate l/min] [
100
u 2 [l/min]
u 1 [l/min]
0
h1
r1
u1
6
u2
4 2 0
0
100
200
300
400
500
600
Tim e [s]
(b) Control actions. Fig. 5. Model-based predictive control. In view of the observed residual static offsets when the system is controlled with a standard MPC technique, the filter (9) was incorporated into the original model-base control system, according to Fig. 1, in order to mitigate this behaviour. Experimental results for α = 150 and taking the same controller’s parameters as those used in the previous experiment are displayed in Fig. 6. As can clearly be observed the extended control system comprising a LIMPC and static offset compensation, by appropriately changing the reference signals supplied to the MPC module is able to drive the system to a zero steady-state offset in a smooth way and in a finite number of time steps, once the system is brought to the vicinity of the true reference by the virtually sole contribution of the LIMPC component. Furthermore, since every single control actions evaluation is performed exclusively within the model-based predictive control structure, being the reference signal vector supplied in advance, violations of constraints are not expected at all.
0.5 0.4 Levelm[]
Acknowledgements
h1
This work was partially supported by the ALCINE and the PRAXIS/P/EEI/14155/1998 programs.
r1
0.3
r2
h2
0.2
References
0.1 0
0
100
200
300 400 Tim e [s ]
500
600
(a) Set-points and outputs. Flow Rate l/min] [
8 u2
u1
6 4 2 0
0
100
200
300 400 Tim e [s ]
500
600
(b) Control actions. Fig. 6: Extended model-based predictive control.
5. Conclusions In this paper an implementation of a pseudo nonlinear black-box model predictive control guaranteeing free offset errors is presented and implemented on a threetanks system. Modelling is based on a non-linear statespace neural network that is trained offline by using the Levenberg-Marquardt algorithm. Given inevitable steady-state offsets arising from model mismatch and degradation stemming from local instantaneous linearisation, it is proposed an extended linear instantaneous model-based predictive control strategy where the outputs are fed back for reference signals manipulation purposes. As errors are weighted by a decreasing exponential function of the error before entering the integrator, only very small errors are in fact accumulated and hence playing a significant role in the offset compensation. Results from experiments show that the proposed extended MPC scheme ensures a quite good tracking performance despite modelling error, as can be seen from the system’s response in the neighbourhood of the set points. Further work should consider robustness, disturbance rejection and global stability analysis.
[1] B. Quin, and M. Badgwell, “An overview of nonlinear model predictive control applications,” Nonlinear MPC Workshop, Ascona, Switzerland, June 2-6, 1998. [2] C. Garcia, D. Prett and M. Morari, “Model predictive control: Theory and practice – a survey,” Automatica, vol. 25, 3, pp. 335-348, 1989. [3] L. Cavagnari, L. Magni and R. Scattolini, “Neural network implementation of nonlinear recedinghorizon control,” Neural Computing & Applications, vol. 8, pp. 86-92, 1999. [4] E. Camacho and M. Berenguel, “Robust adaptive model predictive control of solar plant with bounded uncertainties,” Int. Jour. Adap. Control and Signal Applic., vol. 11, pp. 311-325, 1997. [5] K. Temeng, P. Schnelle and T. McAvoy, “Model predictive control of an industrial packed bed reactor using neural networks,” Jour. Proc. Control, vol. 5, no. 1, pp. 19-27, 1995. [6] C. Bailer-Jones and D. MacKay, “A recurrent neural network for modelling dynamical systems,” Computation in Neural Systems, vol. 9, pp. 531547, 1998. [7] K. Funahashi, “On the approximate realization of continuous mappings by neural networks”. Neural Networks, vol. 2, pp. 183-192, 1989. [8] M. Garzon and F. Botelho, “Dynamical approximation by recurrent neural networks,” Neurocomputing, vol. 29, pp. 25-46, 1999. [9] A. Draeger, S. Engell and H. Ranke, “Model predictive control using neural networks,” IEEE Control Systems Magazine,” vol. 15, no. 5, pp. 6166, 1995. [10] I. Rivals and L. Personnaz, “Black-box modelling with state-space networks,” Neural Adaptive Control Technology, R. Zbikowski and J. Hunt eds., World Scientific, pp. 237-264, 1996. [11] M. Hagan and M. Menhaj, “Training feedforward networks with the Marquardt algorithm,” IEEE Transactions On Neural Networks, vol. 5, no. 6, pp. 989-993, 1994. [12] E. Sontag, “Neural networks for control,” In Essays on Control: Perpectives in the theory and its applications (H. Trentelman and J. Willens, eds.), Birkhauser, Boston, pp. 339-380, 1993. [13] D. Pham and L. Xing, “Neural networks for identification, prediction and control,” SpringerVerlag London Limited, 1995
