On Application Oriented Experiment Design for Closed-loop System Identification AFROOZ EBADAT Licentiate Thesis Stockholm, Sweden 2015 TRITA-EE 2015:002 ISSN 1653-5146 ISBN 978-91-7595-410-3 KTH Royal Institute of Technology School of Electrical Engineering Department of Automatic Control SE-100 44 Stockholm SWEDEN Akademisk avhandling som med tillstånd av Kungl Tekniska högskolan framlägges till offentlig granskning för avläggande av teknologie licentiatesexamen i electro och systemteknik torsdagen den 3 February 2015 klockan 13.00 i Drottning Kristinasväg 30, floor 03 Lantmäteri , KTH Campus. © Afrooz Ebadat, January 2015 Tryck: Universitetsservice US AB iii Abstract System identification concerns how to construct mathematical models of dynamic systems based on experimental data. A very important application of system identification is in model-based control design. In such applications it is often possible to externally excite the system during the data collection experiment. The properties of the exciting input signal influence the quality of the identified model, and well-designed input signals can reduce both the experimental time and effort. The objective of this thesis is to develop algorithms and theory for minimum cost experiment design for system identification while guaranteeing that the estimated model results in an acceptable control performance. We will use the framework of application oriented Optimal Input Design (OID). First, we study how to find a convex approximation of the set of models that results in acceptable control performance. The main contribution is analytical methods to determine application sets for controllers with no explicit control law, for instance Model Predictive Control (MPC). The application oriented OID problem is then formulated in time domain to enable the handling of signals constraints, which often comes from the physical limitations on the plant and actuators. The framework is the extended to closed-loop systems. Here two different cases are considered. The first case assumes that the plant is controlled by a general (either linear or non-linear) but known controller. The main contribution here is a method to design an external stationary signal via graph theory such that the identification requirements and signal constraints are satisfied. In the second case application oriented OID problem is studied for MPC. The proposed approach here is a modification of a results where the experiment design requirements are integrated to the MPC as a constraint. The main idea is to back off from the identification requirements when the control requirements are violating from their acceptable bounds. We evaluate the effectiveness of all the proposed algorithms by several simulation examples. Acknowledgements I would like to express my sincere appreciation towards my supervisor Bo Wahlberg, whose constructive supports and guidance throughout the last years have lead to this work. I am also grateful to my co-supervisors Håkan Hjalmarsson and Cristian Rojas to whom I owe a substantial part of the knowledge of this thesis. I am also immensely grateful to people of automatic control lab for making every day at lab enjoyable. I would like to thank Mariette for all the pleasant time we had together while sharing office and space. I really appreciate the collaborations with her, Christian, Patricio and Per. I am also thankful to Mariette, Christian and Pato for reading my thesis and providing valuable suggestions. A special thank to Patricio and Niklas for the interesting discussions and pleasurable group meetings, trips and courses. I am also grateful to Giulio and Damiano for their vital and persistent supports and good advice. I would like to thank Anneli, Hanna and Kristina for their administrative supports and helps especially when I arrived to Sweden. I thank Swedish Research Council and European Commission for their financial support which made this work possible for me. Finally, I would like to express my appreciation to my family for their love and unconditional support throughout my life and my studies. v Contents Contents 1 Introduction 1.1 Motivating examples . . 1.2 Problem formulation . . 1.3 Related work . . . . . . 1.4 Contributing papers and vi . . . . . . . . . . . . . . . . 1 2 3 5 7 2 Background 2.1 System identification . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Application oriented experiment design . . . . . . . . . . . . . 2.3 Application oriented identification: frequency-domain approach 2.4 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . 2.5 Experiment design for model predictive control . . . . . . . . . . . . . . . . . . . . . . . . 11 11 15 17 20 22 3 Application Set Approximation for MPC 3.1 Problem formulation . . . . . . . . . . . . 3.2 Application set approximation . . . . . . 3.3 Numerical examples . . . . . . . . . . . . 3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 27 29 35 39 4 Time-Domain Input Design 4.1 Problem formulation . . . . . . . . . . . . 4.2 Optimization method: a cyclic algorithm . 4.3 FIR example . . . . . . . . . . . . . . . . 4.4 Numerical results . . . . . . . . . . . . . . 4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 41 46 48 50 54 5 Closed-Loop Input Design 5.1 Problem definition . . . . . . . . . . . . . 5.2 Convex approximation of the optimization 5.3 Fisher information matrix computation . 5.4 Numerical example . . . . . . . . . . . . . 5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . problem . . . . . . . . . . . . . . . . . . . . . . . graph theory . . . . . . . . . . . . . . . . . . . . . . . . 57 59 61 66 69 76 . . . . . . . . . . . . . . . outline vi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . via . . . . . . . . . . . . . . . . . . . . . . CONTENTS 6 Application Oriented Experiment Design for MPC 6.1 MPC with integrated experiment design for OE models 6.2 Closed loop signal generation with back-off . . . . . . . 6.3 Numerical example . . . . . . . . . . . . . . . . . . . . . 6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . vii . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 80 81 83 86 7 Conclusion 89 7.1 Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Bibliography 93 List of Notations 1 (.) 2 Hermitian square root of a positive definite matrix. det(.) Determinant operator. As N (., .) Asymptotic normal distribution. ESI (α) An α-level confidence ellipsoid for the identified parameters through Prediction Error Method (PEM). E {.} Expected value. θ̂N Estimated parameters from N observations. 1 A vector with all elements equal to 1. P Probability measure. Rn Set of n-dimensional vectors with real entries. Rn×m Set of n × m matrices with real entries. GC n n-dimensional de Brujin graph derived from C n . Gν Directed graph with nodes in ν. M Model class. N (x, y) Normal distribution with mean x and variance y ≥ 0. PC Set of probability mass functions associated with stationary vectors. V Set of nodes. VPC Set of all the extreme points of PC . IF Fisher Information Matrix. ⊗ Kronecker product. tr {.} Trace operator. ix x CONTENTS θ Unknown parameters vector. Θapp (γ) The set of acceptable parameters for a control application. θo True parameters vector. Vapp Application cost function. A>B The matrix A − B is positive definite. A≥B The matrix A − B is positive semi-definite. Im Identity matrix of size m. P Cumulative distribution function (cdf). p Probability distribution function (pdf). q Time shift operator. ZN Dataset containing N measurements of input-output samples. Acronyms HVAC Heating, Ventilation and Air Conditioning PI Proportional-Integral MPC Model Predictive Control LTI Linear Time-Invariant PEM Prediction Error Method PRBS pseudo random binary signal OE Output Error ARX Auto Regressive eXogenous LMI Linear Matrix Inequality pdf probability density function pmf probability mass function RMS Root Mean Square SVD Singular Value Decomposition prbs pseudo random binary signal OID Optimal Input Design xi Chapter 1 Introduction Mathematical modeling of real-life systems is of great significance in variety of fields such as chemical process control, biology, building automations, etc. Models are used to describe or predict how systems behave. This in turn, enables us to control the systems and force them to behave as desired. Physical laws and principles can be employed to construct such models. However, accurate mathematical modeling of real-life systems is not always possible. In many cases the existing knowledge of the system is not enough to describe all the properties of the plant. In some other cases the models based on the physical characteristics of the plants are too complicated to be analysed. Thus, there has been an interest in the problem of plant modeling based on experimental data. Preliminary steps in any modeling or system identification based on experimental data are to monitor the system’s behaviour and collect data. The main question that arises is then “How to collect data to obtain as much information as possible about the system, for example in the shortest possible time?” or “How to perform the system identification experiment to generate informative data?”. The effort to answer these questions leads to the growth of the topic of experiment design for identification. In the experiment design problem we design a particular input signal for the system to be modeled, run the experiment, excite the system with the obtained input and record the resulting output signal. The obtained input and output data are used to find an appropriate model for the system. An appropriate input signal reveals the interesting properties of a system in the output while hiding the properties of little or no interest. Thus, the next question that arises is “What are the properties of interest?”. In order to answer this question, one needs to know the intended model application. The experiment should be designed for the intended application. This problem is studied under the topics of identification for control, least costly identification and application oriented experiment design. More elaborate descriptions of system identification can be found in [41], [55] and [21]. 1 2 CHAPTER 1. INTRODUCTION 1.1 Motivating examples To motivate the work presented in this thesis a few examples are presented here. The examples illustrate the need of designing application oriented experiment for system identification while there are physical limitations on the signal values. Example 1.1.1 (Heating, Ventilation and Air Conditioning (HVAC)). (HVAC) Buildings are responsible for about 40% of global energy consumption and more than half of this energy is consumed by HVAC systems [2]. Thus, one can significantly save energy by improving the efficiency of HVAC systems. The main goal in such a problem is to achieve comfortable values of temperature and find appropriate ventilation level with minimum energy consumption. Consider the temperature control problem in an office. The office temperature is tuned based on a heater system. The heater temperature can be controlled with a simple Proportional-Integral (PI) controller. However, to tune such a controller effectively, a model of the heater is required. Such a model can for example be obtained based on the theoretical tools available in system identification field. Figure 1.1: Diagram of one radiator in the heater system. The first step is to identify the inputs and outputs. A heater is a simple radiator connected to a piping system. It is possible to control the heater temperature by the hot water flow to the radiator through an electro-valve1 . Thus, the radiator temperature is affected by the radiator valve. In order to design a controller the relation between the valve opening percentage (input) and the radiator temperature (output) needs to be identified. The next step is to design an experiment to find an exciting enough signal to the radiator. The excitation signal should be such that we get as much information as possible regarding the relation between the valve opening and radiator temperature. However, the input (valve opening percentage) has some physical restriction and it can only vary between 0 and 100%. Moreover, it is not usually possible to detach the radiators to do experiments. The experiment should be performed in closed-loop and 1 For simplicity, we assume that the water temperature is constant. 1.2. PROBLEM FORMULATION 3 people comfort during the experiment should also be taken into account. In other words, during the experiment the room temperature can not violate the constraints. Therefore, in order to design a PI controller for a heating system we need to solve an experiment design problem with several constraints on the input and output levels. Example 1.1.2 (Oil and gas reservoirs). The increasing energy demand due to the growth of human population necessitates recovering the existing hydrocarbon resources in an optimal way. This has led to an emerging interest in reservoir management problem in which reservoir modeling or simulation is a crucial step. The main incentive for reservoir simulation is increased profitability through better reservoir management. A realistic model of reservoir can be an effective tool for developing and evaluating plans to improve oil well’s productivity and ultimate recovery. In reservoir exploration phase, data acquisition is performed by the help of well testing. Well testing is an important but costly and disrupted process to seek a reliable production of oil and gas from wells. Accurate values of water-cut2 , gas-oil ratio3 and productivity index of a well are required to have a successful production optimization. In order to estimate the aforementioned parameters one way is to stimulate the wells by changing the production rate (excitation signal) and measure the resulting downhole pressure (output). However, the flow rate is limited due to the well and rock properties and reservoir pressure (input constraints). On the other hand, the wells should not be interrupted too much during the well test phase due to the production constraints, see [67]. These limitations must be taken into account during the well testing experiment. Therefore, once again we are faced with the problem of designing an optimal flow rate with signal and production constraints. 1.2 Problem formulation The general problem studied in this thesis is the experiment design for system identification. The problem is formulated as designing a least costly input signal for the experiment to get as much information as possible. The obtained information should be aligned with the intended application of the system. Thus, the model application should be taken into account during the experiment design. The model quality for that particular application is evaluated by some quality measure functions. 2 The 3 The ratio of water produced compared to the volume of total liquids produced ratio of produced gas to produced oil 4 CHAPTER 1. INTRODUCTION This thesis considers the experiment design problem when the application of the estimated model is in controller design. Hence, the experiment design problem is defined as an optimization problem. The goal of the optimization problem is to guarantee that the estimated model belongs to the set of models that satisfies the desired control specifications, with a given probability. One should meet this requirement with as little cost as possible. This is the main idea behind the so-called application oriented experiment design. This thesis starts with presenting the existing general framework for application oriented experiment design problem. We then consider one central aspect of experiment design in system identification. As mentioned before, when a control design is based on an estimated model, the achievable performance is related to the quality of the estimate. The degradation in control performance due to errors in the estimated model is measured by an application cost function. In order to use an optimization based input design method, a convex approximation of the set of models that satisfies the control specification is required. The standard approach is to use either a quadratic approximation of the application cost function, where the main computational effort is to find the corresponding Hessian matrix or a scenario based approach where several evaluations of the cost function is required. In this thesis, an alternative approach for this issue is proposed. The method uses the structure of the underlying optimal control problem to compute analytically the required derivatives of the cost with considerably reduced computational effort. The proposed approach is suitable for controllers with implicit control law such as Model Predictive Control (MPC) and problems with large number of parameters. Moreover, in many cases, a second order approximation of the cost function is not very good, especially for low performance demands. The suggested method, however, can compute higher order derivatives at the same time. This makes it possible to use higher order expansions of the application cost function when it is necessary. In reality there are some limitations on input signals and the resulting output signals due to the physical restriction of the system. This issue is also recognized and discussed. A method for application oriented optimal input design under input and output constraints is presented. In this method the corresponding optimization problem for application oriented experiment design is formulated in the time-domain to handle the aforementioned constraints. The method is evaluated on a simulation example. In practical applications, many systems can only work in a closed-loop setting due to stability issues, production restriction, economic considerations or inherent feedback mechanisms. For systems in closed-loop, one needs to take into account that the measurement noise and the input signal are correlated. This fact is also considered in this thesis and application oriented experiment design is extended to identification of the closed-loop systems with known control laws. There are, however, advanced controllers such as MPC for which the control law is not explicitly known. Due to the importance of MPC, the problem of closedloop application oriented experiment design for MPC is studied. A recent idea is to integrate the requirements of the excitation signal to the MPC formulation. A fundamental problem that arises by simply adding this constraint is the risk of 1.3. RELATED WORK 5 having no feasible solution. This issue is studied and addressed by proposing a back-off algorithm. The algorithm seeks for a trade-off between the identification requirements and control performance during the experiment. It is worth mentioning that we consider Linear Time-Invariant (LTI) systems and to solve the identification problem we use the Prediction Error Method (PEM), where we minimize the one step ahead prediction error of the output. 1.3 Related work The problem of input design for system identification has been extensively studied in the literature, see e.g. [41], [21], and [4]. One common approach to formulate this problem, which is used in this thesis, is to define an optimization problem to design an input signal. The input signal should excite the important properties of the system for the intended application of the estimated model. This can guarantee that a certain accuracy is obtained during the identification. The experimental effort to obtain such an accuracy should be minimized. This approach ensures that we are not spending more effort on identification than necessary. The required accuracy is often measured in terms of the application of the obtained identified model. This induces growth of the ideas of identification for control, least costly identification and application oriented experiment design, see [26], [20], [4], and [27]. In application oriented experiment design, the identification objective is to guarantee that the estimated model belongs to the set of models that satisfy the control specifications with a given probability. In [27], this goal is stated mathematically as a set constraint where the set of all identified models corresponding to a particular level of confidence (identification set) must lie inside the set of all models fulfilling the control specifications (application set). In general, the obtained set constraint is not convex and thus one aspect of the application oriented experiment design introduced in [27] is application set approximation. Two known approaches for making a convex approximation of this set are the scenario approach, [36] and [9], and the ellipsoidal approach [66]. In the scenario approach, the application set is described by a finite (but large enough) number of samples which are randomly chosen from the set. In the ellipsoidal approximation, however, a second order Taylor approximation of the cost function is used, which requires Hessian matrix computations. The main drawback of existing methods is that for systems with large number of parameters and controllers without explicit control law, like MPC, both methods require several simulations to be made of the closed-loop system with the controller in the loop. In addition, in [28] it is shown that the second order approximation is not very good in cases with low performance demand and the difference between the second order approximation and higher order approximations can be quite large. Thus, finding a time and cost effective method to compute the derivatives of the chosen application function is of great importance. In the aforementioned formulation of the experiment design problem one needs to measure the quality of the model and find the set of all identified models for 6 CHAPTER 1. INTRODUCTION a particular level of confidence. In [41], it is shown that the inverse of the Fisher information matrix is a lower bound on the covariance matrix for any unbiased estimator. Thus, the Fisher information matrix is used to find the identification set. For model structures linear in input, the information matrix is asymptotically an affine function of the input power spectrum. Therefore, the input design problem is usually formulated in the frequency domain, where the obtained optimization problem can be formulated as a semi-definite program [32]. The outcome is an optimal input spectrum or an autocorrelation sequence. The optimal input values are obtained from the given optimal spectrum, see [15]. The problem is, however, more complex for nonlinear dynamical systems since the input spectrum is not enough to describe the information matrix. Here, the probability density function of the input signals can be optimized instead of the input spectrum. Similar to the linear case, one can generate the time realizations given the probability density function, see e.g., [62] and [16]. In practice there are bounds on the input signals and the resulting output signals, which should be taken into account during the experiment design. These constraints are typically expressed in the time-domain and how to handle this in frequency-domain is not evident. One way to get around this problem is to impose these constraints during the generation of a time realization of the desired input spectrum, see [22], [40], [53] and [35]. There are, however, a few approaches that try to solve the optimal input design problem directly in the time domain, see e.g [44] for linear systems. The main advantage is that in the time domain the constraints on the amplitude of the input and the system dynamics appears naturally and are easier to handle. However, the main difficulty that arises is that the problem is non-convex. For example, for a system with one unknown parameter and input length of n, the number of local optima is 2n [44]. In [44], this problem is addressed through a semidefinite relaxation of quadratic programs and the Fisher information matrix is maximized under some constraints on the input signal. Another way to handle this problem is to seek the optimal transition probabilities of a markov process for the input, see [8]. In Chapter 4 of this thesis we present another approach to get around this problem. Many dynamical systems can only work in closed-loop settings and thus the application oriented experiment should be performed in closed-loop. There is a quite rich literature on closed-loop identification with three main approaches: direct methods (the model is identified as if the system were in open-loop), indirect methods (the model is identified from the identified closed-loop structure), and joint input-output (an augmented model is identified, where the input and output of the system are considered as the new outputs, and the reference and noise as new inputs); see e.g. [41, 55, 17, 65] and the references therein. For closed-loop models, the input design problem is of great importance. Closed-loop experiment design is often translated to design the spectrum of an additive external excitation, where the controller is fixed. The problem is also studied for the cases where the controller can also be designed (provided it is a design variable), see [25, 31, 29, 24] 1.4. CONTRIBUTING PAPERS AND OUTLINE 7 and references therein. However, the main limitation on the existing methods is that they cannot be employed in closed-loop systems with nonlinear feedback. In addition, they cannot handle probabilistic constraints on the input and output, which arise for safety or practical limitations. Application oriented experiment design for MPC MPC is one of the most popular advanced controllers in process industry [46]. The performance of MPC is highly dependant on the quality of the model that is being used. Modeling is one of the most time consuming parts in MPC design [68]. On the other hand, figures illustrate that identified models outperform the physical models in most of the cases. Thus, identification for MPC has become a significant issue. In the literature, pseudo random binary signal (PRBS) excitations are frequently used for the identification process for MPC, see [48]. However, there are some other works that try to find the input excitation signal, optimally. For example, in [36], the authors implement the application oriented experiment design framework presented in [27] on MPC to find optimal input signal. The main challenge in experiment design for MPC is lack of explicit solution for MPC due to input and output constraints and finite horizon optimization. The problem of closed-loop identification method for MPC is also studied in the literature. In [42], the authors proposed an automated identification process to find an appropriate model for MPC among a large group of models. The main idea is that to meet all important conditions in industry, more than one model is required. In [33], again the application oriented experiment design idea has been used for closed-loop identification for MPC. The main idea here is to include excitation requirements in the MPC formulation. The main difficulty is that the obtained optimization problem is not convex. For a special case of Output Error (OE) models a convex relation of the problem is presented. However, for more complicated model structures, it is a challenging problem to predict the information matrix during the prediction horizon of MPC. 1.4 Contributing papers and outline The materials presented in the chapters of this thesis are based on several previously published papers. The organization of the chapters of this thesis and the connections between related publications and the different chapters of the thesis are presented below. Chapter 2 - This chapter summarizes several necessary preliminaries on system identification and application oriented experiment design. Chapter 3 - This chapter considers the experiment design problem for MPC and addresses a common problem that arises due to the lack of explicit solution. This chapter is based on 8 CHAPTER 1. INTRODUCTION A. Ebadat, M. Annergren, C. A. Larsson, C. R. Rojas, B. Wahlberg, H. Hjalmarsson, M. Molander, and J. Sjoberg. Application Set Approximation in Optimal Input Design for Model Predictive Control. In 13th European Control Conference, Strasbourg, France, June 2014. Chapter 4 - A time-domain application oriented experiment design under input and output constraints is presented in Chapter 4. The materials in this chapter are published in the following paper A. Ebadat, B. Wahlberg, H. Hjalmarsson, C. R. Rojas, P. Hägg, and C. A. Larsson. applications oriented input design in time-domain through cyclic methods. In 19th IFAC World Congress, Cape town, South Africa, August 2014. Chapter 5 - The problem of application oriented experiment design for closedloop system identification with a known controller is considered in Chapter 5. This chapter is based on A. Ebadat, P. E. Valenzuela, C. R. Rojas, H. Hjalmarsson, and B. Wahlberg. Applications oriented input design for closed-loop system identification: a graph-theory approach. In IEEE conference on Decision and Control (CDC), Los Angles, US, December 2014. Chapter 6 - In this chapter applications oriented input design for closed-loop systems with MPC in the loop is studied. The results of this chapter are reported in H. Hjalmarsson, C. A. Larsson, P. Hägg, and A. Ebadat. Delivarable 3.3 Novel algorithms for productivity preserving testing. Autoprofit project C. A. Larsson, A. Ebadat, C. R. Rojas, and H. Hjalmarsson. An applicationoriented approach to optimal control with excitation for closed-loop identification. European Journal of Control. Submitted. The following publications are not covered in this thesis, but contain a few related materials and applications: • A. Ebadat, G. Bottegal, D. Varagnolo, B. Wahlberg, and K. H. Johansson. Estimation of building occupancy levels through environmental signals deconvolution. In ACM Workshop On Embedded Systems For Energy-Efficient Buildings, 2013. • P. Hägg, C. A. Larsson, A. Ebadat, B. Wahlberg, and H. Hjalmarsson. Input signal generation for constrained multiple-input multiple output systems. In 19th IFAC World Congress, Cape town, South Africa, August 2014. 1.4. CONTRIBUTING PAPERS AND OUTLINE 9 Contributions by the author The contributions of the above mentioned publications are the outcomes of the author’s own work, in collaboration with the corresponding co-authors. Chapter 5 is the result of close collaboration between the author and Patricio Esteban Valenzuela with the equally shared contributions. In Chapter 6, however, the contribution of the author in the publications is the minimum time controller. Chapter 2 Background In this chapter we present the required materials in system identification, application oriented experiment design and MPC which are used in this thesis. 2.1 System identification System identification is a well-studied field with tools that construct mathematical models to describe the behaviour of dynamical systems. A wide range of system identification applications can be found in almost every single engineering branch. Generally speaking, a primary task in any identification method is to choose a model class. The model is parametrized by some unknown parameter vector 1 . The aim is then to find the unknown parameters such that the selected model can describe the studied dynamic system. The obtained mathematical model should capture the properties of the dynamic system based on available experimental data. System and model In this thesis we focus on the identification of discrete-time multivariate systems that are causal LTI, that is S: y(t) = G0 (q)u(t) + H0 (q)e0 (t), (2.1) where u(t) ∈ Rnu and y(t) ∈ Rny are the input and output vectors, and G0 (q) and H0 (q) are the transfer function matrices of the system. The transfer function H0 (q) is assumed to be stable, inversely stable and monic linear filter. The signal e0 (t) ∈ Rne is white Gaussian noise with E {e(t)} = 0 and E e(t)eT (t) = Λ0 . Let q −1 denote the backward shift operator, i.e., q −1 u(t) = u(t − 1). The schematic representation of system (2.1) is shown in Figure 2.1. 1 An alternative approach is to estimate the impulse response of the system instead of parametrizing the model. This is called non-parametric system identification. However, in this thesis we focus on parametric identification. 11 12 CHAPTER 2. BACKGROUND e(t) H0 (q) u(t) ++ G0 (q) y(t) Figure 2.1: Schematic representation of system (2.1). We first define a model class, M, for the system (2.1). The model is parametrized by an unknown parameter vector θ ∈ Rnθ , that is, M(θ) : y(t) = G(q, θ)u(t) + H(q, θ)e(t), (2.2) where again E {e(t)} = 0 and E e(t)eT (t) = Λ. It is assumed that the model (2.2) perfectly matches system (2.1) when θ = θo . We call θo the true parameter vector. The goal of system identification is to find the value of θ that can describe the system behaviour according to some quality measures. Prediction error method Once the model structure is chosen and parametrized, the next step is to estimate the unknown parameters. To this end, we first collect N samples of the inputoutput data and denote the set of N measurements of the input-output samples by Z N = {u(1), y(1), . . . , u(N ), y(N )}. The available measurements are then used to estimate the unknown parameters θ. The estimated parameter vector, given N measurements in the experiment, is denoted by θ̂N . The method we exploit for parameter estimation is PEM, where the quality of the estimated model is measured based on the error between the true measured outputs and the predicted ones by the model. We use a quadratic cost to evaluate the quality of different models. To proceed, consider the one step ahead prediction of y(t) given the model structure M(θ) ŷ(t|θ) = H(q, θ)−1 G(q, θ)u(t) + [Iny − H(q, θ)−1 ]y(t), (2.3) see [41]. Assuming H(q) to be stable, inversly stable and monic, the predictor is stable and it does not depend on y(t). The prediction error is then (t, θ) = y(t) − ŷ(t|θ) = H(q, θ)−1 [y(t) − G(q, θ)u(t)]. (2.4) The parameter estimation problem is defined as θ̂N = arg min VN (θ, Z N ), θ (2.5) 2.1. SYSTEM IDENTIFICATION where VN (θ, Z N ) = 13 N 1 X T (t, θ)Λ−1 0 (t, θ), 2N t=1 (2.6) is a well-defined and scalar-valued function of θ, see [41] for more details. Statistical properties of PEM Assume that the model set M(θ) contains the true system, i.e., θo exists. Under some mild assumptions, the estimated parameter θ̂N from (2.5) has the following asymptomatic (N → ∞) property: √ N θ̂N − θo ∈ As N (0, Pθ ) , (2.7) where the covariance matrix Pθ is given by " #−1 N 1 X −1 T E ψ(t, θo )Λ0 ψ(t, θo ) Pθ = lim , N →∞ N t=1 (2.8) d ψ(t, θ) = ŷ(t|θ), dθ see [41] for further details. Note that we know N is finite but we can assume it is large enough such that the aforementioned asymptotic properties still hold with good approximation. The covariance matrix Pθ has some useful properties in the frequency domain. Lemma 2.1.1 states the frequency domain expression for P −1 . Lemma 2.1.1. Consider open loop system identification. The inverse of the covariance matrix Pθ−1 defined in (2.8), is an affine function of the input spectrum in frequency domain and is given by Rπ 1 jω −jω,θo T Pθ−1 = 2π Γ (ejω,θo ) Λ−1 ) dω (2.9) 0 ⊗ Φu (e ) Γu (e −π u R π −1 −1 jω 1 jω,θo −jω,θo T + 2π −π Γe (e ) Λ0 ⊗ Λ0 (e ) Γe (e ) dω, (2.10) where vecFu1 vecFe1 Γu = ... , Γe = ... , vecFunθ vecFenθ Fui = H −1 dG(ejω , θ) dH(ejω , θ) , Fei = H −1 , for i = 1, . . . , n. dθi dθi Here Φu (ejω ) is the spectrum of the input signal. Note that the operator vecX returns a row vector shaped by putting the rows of matrix X, after each other. Proof. See Lemma 3.1. in [3, pp. 39-40]. 14 CHAPTER 2. BACKGROUND System identification set We can use lemma 2.1.1 to find an α-level confidence ellipsoid for the identified parameters using PEM as follows: ESI (α) = T θ : [θ − θo ] Pθ−1 [θ − θo ] ≤ χ2α (nθ ) N , (2.11) where χ2α (nθ ) is the α-percentile of the χ2 -distribution with nθ degrees of freedom. We thus have that θ̂N ∈ ESI (α) with probability α when N is large enough. The set ESI (α) is called identification set. Example 2.1.1 (System identification set). Consider HVAC system in Example 1.1.1. Assume that we aim to identify a model from the radiator valve to the radiator temperature. We consider the following model structure: M(θ) : y(t) = θ1 y(t − 1) + θ2 u(t − 1) + e(t), (2.12) where y(t) is the radiator temperature and u(t) is the valve opening percentage and E {e(t)} = 0, E e(t)eT (t) = Λ0 . A parameter vector θ0 exists such that S = M(θ0 ). Based on (2.3), the one step ahead prediction of the output is ŷ(t|θ) = θ1 y(t − 1) + θ2 u(t − 1). (2.13) The inverse of the covariance matrix Pθ is then obtained by N 1 X E {y(t − 1)y(t − 1)} E {u(t − 1)y(t − 1)} N →∞ N Λ0 t=1 Pθ−1 = lim E {y(t − 1)u(t − 1)} . E {u(t − 1)u(t − 1)} (2.14) For assume that the input sequence {u(t)} is white with zero mean and simplicity E u2 (t) = Ruu . The limit (2.14) thus exists and is Pθ−1 1 = Λ0 " θ22 R 1−θ12 uu + 0 1 Λ 1−θ12 0 # 0 . Ruu (2.15) The identification set is then obtained by substituting (2.14) in (2.11). 2.2. APPLICATION ORIENTED EXPERIMENT DESIGN 15 Cramér-Rao inequality The Cramér-Rao inequality expresses a lower bound on the values of the mean square error between the estimated parameters and the true parameters, i.e. o n E (θ̂N − θo )(θ̂N − θo )T ≥ I−1 (2.16) F (θo ), where IF is the Fisher information matrix and is defined by ( ) T ∂ log p(y N |θ) ∂ log p(y N |θ) IF (θo ) := E ∂θ ∂θ , (2.17) θ=θo with y N = {y(1), . . . , y(N )} and p(y N |θ) is the probability density function of y N given the parameters θ, see [41, pp.245-246] for the proof. Regarding the asymptotic properties of the PEM framework, we can conclude that for large values of N the covariance of the estimates is equal to the Cramér-Rao bound, N cov(θ̂N − θo ) = Pθ = N I−1 (2.18) F (θo ). 2.2 Application oriented experiment design Usually any identification problem is followed by another problem which makes use of the identified model. One major application of system identification is controller design. The performance of a controller designed based on a model is potentially affected by the model quality. Thus, to satisfy minimum requirements on the control performance, a quality constraint should be imposed on the estimated model. This can be obtained by designing an appropriate input signal for the identification experiment, i.e. optimal input design. Optimal Input Design (OID) in system identification has been extensively investigated and formulated in several different forms, see e.g. [41], [21] and [4]. One common idea is to design an input signal such that a certain accuracy is obtained during the identification and simultaneously minimize the experimental effort to obtain such an accuracy. In this thesis we consider the case where the accuracy is defined in terms of the application of the model and the control performance. This is the main idea behind application oriented experiment design [27]. Note that identification for control and least costly identification are also based on the notion of minimizing the experimental effort under constraints on plant-model mismatch (see e.g. [26], [20], [4]). Application cost function A plant-model mismatch is the main cause of performance deterioration of modelbased controllers. We use the concept of an application cost to relate the performance degradation to plant-model mismatch. Consider again the system introduced in (2.1) and the model set M(θ). The application cost is a scalar function of the 16 CHAPTER 2. BACKGROUND parameters. The minimum value of this function should occur for true parameters, θo . For any other parameter vector θ the performance will degrade or remain the same and thus application cost returns higher values. We denote the application function Vapp (θ). In particular, we assume without loss of generality that Vapp (θo ) = 0. We assume the function is twice differentiable in the neighbourhood of θo . This assumption implies that 0 00 Vapp (θo ) = 0, Vapp (θo ) = 0, and Vapp (θo ) ≥ 0, (2.19) see [66] for examples of application function. Application requirements The application function measures the performance degradation of the controller as a result of plant-model mismatch. However, in any application the minimum requirements on the control performance exist. These requirements can be considered by imposing an upper bound on the application function. Based on these premises, we define an application set. The application set contains the acceptable parameters from the control point of view. The idea of defining admissible set in terms of associated control performance was first introduced by [4]. In [27], the authors used similar idea and defined application set. We also use the notion of the application set on this thesis. The set is defined as 1 Θapp (γ) = θ : Vapp (θ) ≤ , (2.20) γ where γ ∈ R is a positive user-defined constant. Every parameter vector θ for which the control performance deteriorates less that γ1 , lies inside the application set and will be considered as an acceptable parameter. Application oriented optimal input design problem The application oriented optimal input design problem is defined as an optimization problem. The objective function of the optimization is a function that measures the cost of performing the identification experiment, for example input power, input energy or experiment time. Moreover, the input signal is designed such that the estimated model guarantees acceptable control performance when used in the control design, that is, it requires that θ̂N ∈ Θapp (γ). Thus a natural way to formulate the optimization problem is minimize Experimental Cost subject to θ̂N ∈ Θapp (γ). input (2.21) As mentioned in Section 2.1, θ̂N is a stochastic variable and consequently is impossible to enforce deterministic bounds on it. We can only satisfy the constraint 2.3. APPLICATION ORIENTED IDENTIFICATION: FREQUENCY-DOMAIN APPROACH 17 with a certain probability. We thus replace the constraint in problem (2.21) by ESI (α) ⊆ Θapp (γ), which ensures θ̂N ∈ Θapp (γ) with probability α, see [51] for other formulations of the problem. The problem can then be formulated in either frequency-domain where the input spectrum is designed, or time-domain where the optimization is performed on the time-domain realizations of the input signal. 2.3 Application oriented identification: frequency-domain approach From Lemma 2.1.1 we know that for a linear system, Pθ−1 is an affine function of the input spectrum in open loop systems and we can design our estimates by designing the input spectrum. This choice of decision variable simplifies the problem greatly and enables the convex approximations of the problem. Therefore, a wide range of input design problems consider the input spectrum Φu as the decision variable, instead of the input signal. The application oriented optimal input design problem in frequency-domain can be formulated as follows minimize Φu Experimental Cost, subject to Φu (ω) ≥ 0, for all ω, ESI (α) ⊆ Θapp (γ). (2.22a) (2.22b) (2.22c) However, the optimization problem (2.22) may not be convex due to the two constraints. The first constraint, spectrum constraint, involves infinitely many inequalities since it should be fulfilled for all ω. The second constraint, set constraint, may not be convex since while the system identification set is an ellipsoid (see (2.11)), the application set can be of any shape. We discuss each of these constraints in more details and we find a convex approximation of them. The obtained approximated problem is convex and thus tractable. Spectrum constraint Consider a stationary signal u(t). The spectral density is given by Φu (ω) = k=∞ X ck ejωk , (2.23) k=−∞ u where ck ∈ Rnu ×n are the autocorrelation coefficients of the input signal, ck = T E u(t)u (t − k) and ck = c−k . Thus, finding the optimal input spectrum in optimization problem (2.22) is equal to computing infinitely many coefficients, which is not computationally possible without furthure structure. To overcome this problem we can use finite dimensional parametrization introduced in [32] with the following 18 CHAPTER 2. BACKGROUND truncated sum k=m−1 X Φu (ω) = ck ejωk . (2.24) k=−(m−1) Consider again the optimization problem (2.22). The constraint (2.22b) should hold for all ω and hence it is an infinite dimensional constraint. However, this constraint can be converted to a Linear Matrix Inequality (LMI) constraint instead, by using the following lemma. Lemma 2.3.1 (Kalman-Yakubovich-Popov). Let {A, B, C, D} be a controllable Pk=m−1 state space realization of k=0 ck ejωk . Then, there exists a Q̄ = Q̄T such that Q̄ − AT Q̄A −AT Q̄B 0 CT K(Q̄, {A, B, C, D}) , + ≥ 0, (2.25) C D + DT −B T Q̄A −B T Q̄B if and only if Φu (ω) = k=m−1 X ck ejωk ≥ 0, for all ω. k=−(m−1) Proof. This lemma is a result of the Positive Real Lemma. For an elaborated proof of this lemma we refer the reader to [6] and the references there in. Application set constraint Consider the application set constraint (2.22c). The identification set (2.11) has an ellipsoidal shape, but the application set can be of any shape. Therefore, this constraint may not be convex and usually a convex approximation is required. This issue is discussed in more details in [51]. In this section, two known approaches to make a convex approximation of the constraint are presented. The two methods are the scenario approach, see [36], [9], and the ellipsoidal approach, see [66]. Scenario approach The constraint (2.22c) can be interpreted as an infinite number of constraints since every point inside ESI is required to be inside Eapp . However there are limited number of decision variables and thus the problem is semi-infinite. In the scenario approach, the application set is described by a finite number, Nk , of samples (or scenarios) which are randomly chosen from the set. The constraint (2.22c) is then replaced by a set of inequalities, [θk − θ0 ]T Pθ−1 [θk − θ0 ] ≥ k γχ2α (n) Vapp (θk ), k = 1, . . . , Nk . N (2.26) In order to have a good approximation of the set constraint, the number of scenarios must be large enough (see e.g. [10] for a lower bound on the number of scenarios). The idea of approximating a semi-infinite problem with a finite optimization 2.3. APPLICATION ORIENTED IDENTIFICATION: FREQUENCY-DOMAIN APPROACH 19 problem is presented in [9]. The implementation of scenario approach for convex approximation of (2.22c) was first proposed in [36]. Remark 2.3.1. The scenario approach requires several evaluation of the application function. In the case of high dimensional and complex plants and controller with implicit control law such as MPC, it is not possible to find analytic expressions for Vapp . Thus, the evaluation necessitates a large number of often highly timeconsuming and costly simulations, which is not applicable in some real processes. Ellipsoidal approach The ellipsoidal approach is based on a second order Taylor expansion of Vapp (θ) around θo , that is, 0 Vapp (θ) ≈ Vapp (θo ) + Vapp (θo )[θ − θo ] 1 00 + [θ − θo ]T Vapp (θo )[θ − θo ] 2 1 00 = 0 + 0 + [θ − θo ]T Vapp (θo )[θ − θo ]. 2 The application set can thus be approximated by 00 2 Eapp (γ) = θ : [θ − θo ]T Vapp (θo )[θ − θo ] ≤ . γ (2.27) (2.28) The application set constraint (2.22c) is N γ 00 P −1 ≥ Vapp (θo ), χ2α (nθ ) θ 2 (2.29) The quality of the approximation not only depends on the application cost but also on the value of γ. For sufficiently large values of γ, Eapp gives an acceptable approximation. For low performance demand, however, the difference between the second order approximation and the realk set can be quite large and higher order approximations are required. This problem is studied in [28] and it is shown that in some cases higher order approximations are necessary and the approximate set constraints are polynomial in the coefficients of the spectrum parametrization. 00 Remark 2.3.2. The calculation of the Hessian matrix, Vapp (θo ), is a challenging task. In many problems it is not possible to analytically determine the Hessian of the application function due to non-linearities in the controllers that are being used. Therefore, numerical approximations are used. Numerical methods, such as finite difference approximation, are not applicable in most cases because of the large number of variables involved. Remark 2.3.3. In the reminder of this thesis (2.29) will be referred to the experiment design constraint. 20 CHAPTER 2. BACKGROUND The obtained approximation of input design problem Based on the approximation of the spectrum constraint and the region constraints, we will come up with the following two optimization problems 1. Scenario-based approximation: minimize Q,c0 ,...,cm−1 s.t. Experimental Cost, K(Q, {A, B, C, D}), (2.30) Q = QT ≥ 0, [θk − θ0 ]T Pθ−1 [θk − θ0 ] ≥ k γχ2α (n) Vapp (θk ), k = 1, . . . , Nk . N 2. Ellipsoidal approximation: minimize Q,c0 ,...,cm−1 s.t. Experimental Cost, K(Q, {A, B, C, D}), Q = QT ≥ 0, N γ 00 Pθ−1 ≥ Vapp (θo ). 2 χα (nθ ) 2 (2.31) The outcome of the obtained optimization problems is the truncated sum (2.24) which gives the signal spectrum. Thus, it is naturally followed by another problem where the main goal is to generate the optimal input values from the given spectrum, see [15]. In practice, however, there are bounds on the amplitude of the excitation signals in real processes and how to handle these constraints in the frequency domain is not evident. In time-domain the aforementioned constraints appear naturally, however, the obtained optimization problem (2.21) is non-convex in time-domain. In Chapter 4 of this thesis we reformulate the optimization problem and we employ the existing optimization techniques to find an optimal solution for the obtained problem. 2.4 Model Predictive Control MPC, also known as receding horizon control or moving horizon control, is an optimization based control technique where a model is used to predict the behaviour of the plant. There is an emerging interest to use MPC in different fields, especially, in industrial process control, where it was used for the first time. One prominent property of MPC is the possibility of handling input and output constraints. The input constraints usually arise because of actuator saturations while the output signal is restricted to keep the plant in the safe region and produce a high quality product. 2.4. MODEL PREDICTIVE CONTROL Past 21 Future Output trajectory 𝑢𝑘 Input trajectory 𝑢𝑘+1 𝑘 𝑘+1 𝑘 + 𝑁𝑢 𝑘 + 𝑁𝑦 Control Horizon Prediction Horizon Optimal trajectory at time k Optimal trajectory at time k+1 Figure 2.2: Receding horizon principle MPC works based on the receding horizon principles. The main idea is to start with a fixed horizon optimal control problem where the future constraints are taken into account during the control design. This results in an optimal control sequence. Form the obtained sequence, only the first control action is implemented on the system. The plant status is updated by measuring the states and outputs of the system after applying the obtained control action. The fixed horizon optimization problem is repeated for the current states. This idea is shown in Figure 2.2. The main advantage of the receding horizon principle is that it can deal with unexpected events in the future. A common formulation of MPC is minimize J(t) = U (t) subject to where Ny X kŷ(t + i|t) − r(t + i)k2Q + i=0 ŷ ∈ Y, u ∈ U. Nu X i=0 k∆u(t + i)k2R , (2.32) 22 CHAPTER 2. BACKGROUND • Nu is the control horizon that defines the number of input samples that are used in the optimization, • Ny is prediction horizon that defines the number of output samples that are predicted, • U (t) = u(t), u(t + 1), . . . , u(t + Nu ) is the input sequence over the control horizon, • ŷ(t+i|t) is the predicted output at time t+i based on the available information at time t, • r(t) is the reference signal, • Q and R are adjustable weight matrices and kxk2A = xT Ax, • Y and U are the output and input constraint sets, respectively, • ∆u(t) is the input change at time t, i.e., ∆u(t) = u(t) − u(t − 1). MPC requires the ability to predict future outputs and control signals. As a consequence, the performance of MPC is highly dependant on the model that is being used for prediction. Any plant-model mismatch may deteriorate the control performance, such as reference tracking, stability and constraint satisfaction. Although many different types of MPC have been developed to deal with such problems, having a good model is still critical in any MPC application. 2.5 Experiment design for model predictive control Controller design based on a model of a system necessitates putting significant efforts on modelling. The modeling is often done by using system identification techniques. In Sections 2.2 and 2.3 we explained the building blocks for designing experiments for identification such that we obtain an appropriate model for a specific control application. In this section we study a specific case, where the estimated model will be employed in MPC. As mentioned in Section 2.4, the input and output constraints are taken into account in MPC. Although this property makes MPC popular in many different applications, in the experiment design problems this can be considered as a limiting factor. This is due to the fact that there is no explicit solution to the MPC optimization problem when input and output constraints are considered [43]. Several problems arise with the practical implementation of the general framework of application oriented experiment design on MPC: 1. The application oriented experiment design problem relies on the knowledge of the true system parameters. However, the true parameters are unknown during the experiment. In order to get around this problem, two different approaches have been used in the literature: 2.5. EXPERIMENT DESIGN FOR MODEL PREDICTIVE CONTROL 23 a) One way to address this problem is to implement a robust experiment design scheme (see [52] for example). The main idea is that the designed input should be robust to the parameter changes; b) Another approach is trying to solve this problem through adaptive procedures. The problem starts from the best available estimation of the parameters and the calculations of the Hessian of the cost function and output predictions are updated as more information is being collected from the system, see e.g. [19]. 2. Finding an appropriate application cost function is a challenging problem for MPC. A good choice of such a function is still an open question. The main purpose of the application cost is to relate the control performance to the model parameters. More importantly, a good application function should point up the directions in the parameter space, for which the control performance is sensitive to the parameter changes. The detected directions are those that should be excited more during the identification process. Thus, an appropriate choice of cost function is highly dependent on the MPC application. If there is a reference trajectory that the one wants to follow by MPC, then an appropriate choice for the cost function could be a measure of the difference between the obtained output trajectory of the system when the true system is employed and the resulting output trajectory for the estimated model, i.e.: Vapp (θ) = N 1 X ky(t, θ) − y(t, θo )k22 , N t=1 (2.33) where y(t, θo ) is the output using the true parameters and y(t, θ) is the resulting output based on the estimated parameters. This cost function is common in the literature of experiment design for MPC, see for example [66] and [33], where the function is computed over a step response of the system with MPC. It is of great significance that the chosen cost function detects the important factors in the intended application, otherwise the designed input based on this cost might excite the system in a way that is not useful for the application. Thus, the cost function is very different from one case to another and evaluating (2.33) based on the step response of the system might not be a good choice in many cases. 3. The last but not the least important problem is approximating the application set. Calculating the two approximate descriptions of the application set presented in Section 2.3 is a challenging problem for MPC. The main difficulty is that there is no explicit solution for MPC due to the input and output constraints. Thus, evaluating the application cost and Hessian computation require a large number of time-consuming simulations and numerical approximations. 24 CHAPTER 2. BACKGROUND Example 2.5.1 (Application set approximation for MPC). Consider the system x(t + 1) = θo2 x(t) + u(t), (2.34) y(t) = θo1 x(t) + e(t), where θo1 and θo2 are not known. The goal is to identify a model of the system (2.34). The model will be used to design MPC. We aim to use MPC with the following cost function to track a reference trajectory for output J(t) = Ny X kŷ(t + i|t) − r(t + i)k2Q + i=0 Nu X k∆u(t + i)k2R , (2.35) i=0 where Ny = Nu = 5, Q = 10, R = 1 and the reference signal r(t) is shown in Figure 2.3 (see Section 2.4 for detailed description of MPC). The input and output constraints are expressed as lower and upper bounds on the signal values, that is |u(t)| ≤ 1, |y(t)| ≤ 2. (2.36) 2 r(t) 1 0 −1 −2 1 2 3 4 5 6 7 8 9 10 t Figure 2.3: Reference trajectory for closed-loop output. To proceed, we first define the application cost function. A simple choice of the cost function which also satisfies (2.19), is the difference between the closedloop output when the true parameters have been used to tune the controller and the closed-loop output when other parameters have been used, i.e., (2.33). Consider the application set (2.20), with γ = 1000. The corresponding level curve together with a number of scenarios, uniformly distributed over Θapp (γ), and the ellipsoidal approximation of the set is shown in Figure 2.4. 2.5. EXPERIMENT DESIGN FOR MODEL PREDICTIVE CONTROL εapp 1.1 1.1 1 1 θ (2) θ (2) 25 θ0 0.9 θ0 0.9 0.8 0.8 0.7 0.7 Θapp Θapp 0.6 0.4 0.5 0.6 θ (1) 0.7 0.6 0.4 0.8 0.5 0.6 θ (1) 0.7 0.8 Figure 2.4: Level curve (‘ ’) corresponds to γ = 1000 for the application cost of system (2.34) in Example 2.5.1. Left: the ellipsoidal approximation of the set (‘ . .’). Right: different scenarios sampling uniformly from the set(‘×’). 0.1 1.3 1.3 0.09 1.2 1.2 0.08 1.1 1.1 1 1 0.9 0.9 0.8 0.8 0.7 0.7 0.02 0.6 0.6 0.01 0.5 0.2 0.07 0.06 0.05 0.04 0.03 0.4 0.6 0.8 1 0.5 0.2 0.4 0.6 0.8 1 Figure 2.5: Left: level curves for the step function as reference trajectory. Right: level curves for the reference signal shown in Figure 2.3. 26 CHAPTER 2. BACKGROUND We can now change the reference trajectory to a step function. The level curves for different values of γ for both references are shown in Figure 2.5. It can be seen that the shape and size of application set is highly affected by the reference trajectory. In the above example, for the ellipsoidal approximation, the Hessian of Vapp is computed numerically using DerivestSuit in Matlab [13]. Numerical calculation of Hessian requires several evaluation of the application cost, which in turn needs several simulations of MPC. On the other hand, in the scenario approach the size and shape of the application set play an important role in the sampling, but they are not known in advance since they can be affected by many different factors. Hence, in practice the Hessian of Vapp is used when sampling from the set. Thus, the Hessian computation is unavoidable in many practical cases regardless of the approximation method that is being used. The problem of Hessian calculation is studied in Chapter 3 of this thesis, where we propose an analytic method to calculate the Hessian and higher order derivatives of the application cost. Chapter 3 Application Set Approximation for MPC MPC has drawn much attention in control field, due to its ability to cope with system constraints. Using MPC, we can deal explicitly with both input and output constraints during the controller design and implementation. The most time and cost consuming part in industrial MPC applications is the modeling part [69]. Therefore, optimal input design for MPC is of great importance. However, the resulting solutions of MPC are difficult to deal with due to the aforementioned constraints. This makes it unavoidable to deploy numerical calculations for the approximation of the application set (2.20) (see [34]). However, numerical methods are time and cost demanding specially for the systems with large number of parameters. In this chapter we present a new approach based on analytical methods to find an ellipsoidal approximation of the application set for MPC. In particular, we focus on OE systems. 3.1 Problem formulation We consider MPC for a subset of M(θ) with H(q, θ) = I, that is, a class of systems with OE structures: y(t) = G(q, θ)u(t) + e(t). (3.1) MPC for OE systems Let {A(θ), B(θ), C(θ)} be an observable state space realization of system (3.1), that is G(q, θ) = C(θ)(qI − A(θ))−1 B(θ), 27 (3.2) 28 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC where C(θ) ∈ Rny ×nx , A(θ) ∈ Rnx ×nx and B(θ) ∈ Rnx ×nu . We study the following MPC formulation for OE systems: minimize J(t, θ) = u {u(t+i,θ)}N i=0 subject to Ny X kŷ(t + i, θ|t) − r(t + i)k2Q + i=0 Nu X k∆u(t + i)k2R , i=0 x̂(t + i + 1, θ|t) = A(θ)x̂(t + i, θ|t) + B(θ)u(t + i, θ), ŷ(t + i, θ|t) = C(θ)x̂(t + i, θ|t), i = 0, . . . , Nu (3.3) ymin ≤ ŷ(t + i, θ|t) ≤ ymax , i = 0, . . . , Nu , umin ≤ u(t + i, θ) ≤ umax , i = 0, . . . , Ny , x̂(t, θ|t) = x∗ (t, θ), ∆u(t, θ) = u(t, θ) − u∗ (t − 1, θ), u∗ (t − 1, θ) is the optimal input value applied to the system at time instant t − 1, and x∗ (t, θ) is the estimated system state at time t, obtained form either a direct measurement or an observer. For simplicity we assume Nu = Ny . We also consider the case where we have only upper and lower bounds on our input and output signals. The values ymax ∈ Rny and umax ∈ Rnu are the maximum allowed values for the output and input signals, respectively, and ymin ∈ Rny and umin ∈ Rnu are the minimum bounds on these signals. For a more detailed description of the above MPC formulation, see Section 2.4. Application cost function for MPC Consider the optimal input design problem (2.22). In order to deploy this formulation, we fist need to define an appropriate cost function Vapp (θ), for MPC. The application cost function measures the amount of performance degradation that stems from plant-model mismatch, see Section 2.2 in Chapter 2. One reasonable choice of application cost function for MPC is the difference between the measured outputs when the controller is running based on the true parameters, θo , and when it is working based on the perturbed parameters θ, that is, M 1 X Vapp (θ) = ky(t, θo ) − y(t, θ)k2 , (3.4) M t=1 where M is the number of measurements used, and the second argument of y is the parameters used in the MPC to compute the optimal control action and y is the corresponding measured output. The application cost (3.4) is one of the common cost functions that has been used in the literature, see e.g. [36, 66]. Although the requirements on the application function that is, (2.19), are met by this choice, some challenges with implementing optimal input design using this cost can be found: • The required knowledge of true system: The application function introduced in (3.4), depends on the true parameter values, which are not available dur- 3.2. APPLICATION SET APPROXIMATION 29 ing the experiment design process. One solution is to replace θo by its best available estimation, θ̂N . As more and more data are collected the estimation will be updated and it asymptotically converges to the true value with probability 1. Note that there is no general proof for this claim, however, for a few specific cases the convergence has been proved. A convergence proof for Auto Regressive eXogenous (ARX) structure models can be found in [19]. Some other discussions with similar ideas can also be found in [39], where this problem has been studied in an adaptive input design setting. • Application cost evaluation: The application cost should be computed for different parameters. In other words, the system behavior should be investigated for different model parameters in the controller design. This is not possible in real processes due to safety issues, physics of the system and the time constraints. Therefore, we consider a simulation based approximation of (3.4), instead M 1 X Vbapp (θ) = ky(t, θ̂N , θ̂N ) − y(t, θ, θ̂N )k2 , M t=1 (3.5) where θ̂N is the best available estimate of θo and the values of y come from simulations; the second argument of y is the parameter used in the MPC formulation (2.32) for predictions, while the third argument of y is the parameter used in lieu of θo . The values of θ̂N may be updated as better estimates are available. In the rest of this chapter we focus on an application set approximation for (3.5), however, the proposed approach can be extended to other choices of application costs. 3.2 Application set approximation In order to obtain a convex approximation of the application set, we employ the ellipsoidal approach. Thus the first step is to compute the Hessian of the application cost. For the application function (3.5), we can calculate the Hessian matrix in terms of the derivatives of y as follows: M oT n ∂y(t, θ, θ̂ ) o 00 2 X n ∂y(t, θ, θ̂N ) N Vbapp (θ̂N ) = |θ=θ̂N |θ=θ̂N . M t=1 ∂θ ∂θ (3.6) θ̂N ) However, finding ∂y(t,θ, is a challenging problem. ∂θ The output signal depends on the input, which comes from the controller. Consequently, in order to find the derivatives in (3.6), the derivatives of the input signal generated by MPC with respect to θ, are required. This is a challenging problem since there is no explicit solution for MPC when there are inequality constraints on input and output signals. 30 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC The proposed solution here is to notice that, when θ is a small perturbation of θ̂N , the active constraints are the same as when the MPC is running based on θ̂N . Thus, the main idea is to let MPC run based on θ̂N at each time instance t, and determine the optimal value of the input signal u(t, θ̂N ). We assume that the active constraints remain the same for small enough perturbations of θ̂N , see [14]. This fact holds whenever the dual variables associated with the last two sets of constraints in (2.32), which are active, are all non-zero. This fact, in turn, holds with probability 1. In addition, under this condition y is differentiable with respect to its second argument, see [5]. Therefore, at time step t, we are able to find an explicit solution of MPC for θ = θ̂N + δθ by considering active constraints as equality constraints. We can analyse the effects of perturbing the parameters when δθ is small enough. In the rest of this section, we briefly describe the explicit solution of MPC when we are considering only active constraints, then we provide insights into the perturbation analysis for the MPC solution. Finally, we show how these concepts can be used to find the derivatives in (3.6) and compute the application cost function. Explicit Solution of constrained MPC Consider the MPC formulation (3.3) at time instant t. In this section, we seek to rewrite the MPC formulation as a quadratic program where we are considering only active constraints obtained by solving MPC for θ̂, which are equality constraints. We first introduce some notations and new variables: X(t, θ) = [x̂(t + Nu , θ|t)T , . . . , x̂(t, θ|t)T , u(t + Nu , θ)T , . . . , u(t, θ)T ]T , (3.7) T Hr = r(t + Nu )T , . . . , r(t)T , 0, . . . , 0 , T Hu = [0, . . . , 0, 1] , H = Hr + Hu u∗ (t − 1, θ), (3.8a) (3.8b) (3.8c) where 1 is a vector of an appropriate size with all elements equal to 1, Inu .. ∆= . 0 0 −Inu .. . · .. . · · I nu 0 0 .. . , −Inu I nu (3.9) where Im indicates an identity matrix of size m and 0 denotes a matrix of an appropriate size with all its elements equal to zero. Q= INu +1 ⊗ Q 0 0 INu +1 ⊗ R , (3.10) 3.2. APPLICATION SET APPROXIMATION I ⊗ C(θ) Υ(θ) = Nu +1 0 31 0 , ∆ (3.11) where by Im ⊗ M , we mean the Kronecker products of Im and M [38]. Consider again (3.3), we can now rewrite the cost function and constraints in the MPC optimization problem invoking the above mentioned variables: Cost function J = (Υ(θ)X(t, θ) − H)T Q(Υ(θ)X(t, θ) − H). (3.12) We append the last equality constraint, ∆u(t, θ) = u(t, θ) − u∗ (t − 1, θ), to the cost function. Constraints on system dynamic C(θ)X(t, θ) = D(θ), (3.13) with I −A(θ) . . . .. .. .. . . C(θ) = . 0 0 ... 0 0 ... −B(θ) . . . .. .. . . I −A(θ) 0 ... 0 I 0 ... 0 .. . 0 .. . 0 .. . , −B(θ) 0 (3.14) D(θ) = Dx (θ)x̂(t, θ|t), T Dx = 0 . . . 0 Inx . Notice that the initial condition on the states is attached to the dynamic of the system. Input-output bounds INu +1 ⊗ C(θ) 0 INu +1 ⊗ ymax −INu +1 ⊗ C(θ) 0 X(θ) ≤ −INu +1 ⊗ ymin . INu +1 ⊗ umax 0 I −INu +1 ⊗ umin 0 −I (3.15) By defining appropriate notations, (3.15) can be written as J (θ)X(θ) ≤ L. (3.16) Finally, we have the following equivalent MPC formulation: minimize (Υ(θ)X(t, θ) − H)T Q(Υ(θ)X(t, θ) − H) X(t,θ) P(t) : subject to J (θ)X(θ) ≤ L(t), C(θ)X(θ) = D(θ). (3.17) 32 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC Suppose that we can solve (3.17) at time t for θ = θ̂N . Let I(t, θ̂N ) be a time varying diagonal matrix, where each diagonal element corresponds to one of the inequality constraints in (3.15). A diagonal element is zero if its corresponding constraint is inactive at time t when the problem is solved for θ̂N and it is one for active constraints. Multiplying (3.15) by I(t, θ̂N ) and introducing INu +1 ⊗ ymax INu +1 ⊗ C(θ) 0 −INu +1 ⊗ C(θ) 0 , L (t) = I −INu +1 ⊗ ymin , Ja (t, θ, θ̂N ) = I INu +1 ⊗ umax 0 I a 0 −I −INu +1 ⊗ umin (3.18) we obtain Ja (t, θ, θ̂N )X(θ) = La (t), (3.19) which represents those inequality constraints that are active at time instance t. We perturb the parameters to θ = θ̂N + δθ, where δθ is small enough such that the active and inactive constraints remain the same. Thus, we have the following problem in lieu of (3.17) (Υ(θ)X(t, θ) − H)T Q(Υ(θ)X(t, θ) − H) C(θ) D(θ) X(θ) = . subject to La (t) Ja (t, θ, θ̂N ) minimize X(t,θ) (3.20) Problem (3.20) is a quadratic optimization problem with affine equality constraints. Thus, the problem is convex. If X ∗ (θ) and λ∗ = (λ∗1 , λ∗2 ) (λ1 and λ2 are Lagrange multipliers) satisfy the KKT conditions, then they are primal and dual optimal, with zero duality gap [7]. The KKT conditions for this problem are 2Υ(θ)T Q(Υ(θ)X ∗ (t, θ) − H) + C(θ)T λ∗1 + Ja (t, θ, θ̂N )T λ∗2 = 0, C(θ) D(θ) ∗ X (θ) = , La (t) Ja (t, θ, θ̂N ) (3.21a) (3.21b) see [7]. This can be written as ∗ X (t, θ) Ψ(θ) λ∗1 = Λ1 (θ)(Hr + Hu u∗ (t − 1)) + Λ2 (θ)(B0 + Bx x̂(t, θ|t), λ∗2 (3.22) 3.2. APPLICATION SET APPROXIMATION 33 where 0 . . . . . . , B0 = 0 Ψ(θ) 0 , . . . . . . 0 La (t) T T 0 0 2Υ(θ) Q . . . . . . . . . . . . ...... 0 Bx = Dx , Λ1 (θ) = , Λ2 = I . . . . . . . . . . . . . ...... I 0 0 2Υ(θ)T QΥ(θ) C(θ)T C(θ) 0 = Ja (t, θ, θ̂N ) 0 JaT (t, θ, θ̂N ) (3.23) The solution of (3.22) is then ∗ X (t, θ) λ∗1 = Ψ−1 (θ) Λ1 (θ)(Hr + Hu u∗ (t − 1)) + Λ2 (B0 + Bx x̂(t, θ|t) . λ∗2 (3.24) Solving (3.24), we get an explicit solution for u(t, θ) as u(t, θ) = Lx (θ)x̂(t, θ|t) + Lu (θ)u∗ (t − 1, θ) + Lr (θ), (3.25) with Lx (θ) = 0, . . . , I, . . . , 0 Ψ−1 (θ)Λ2 Bx , Lu (θ) = 0, . . . , I, . . . , 0 Ψ−1 (θ)Λ1 Hu , Lr (θ) = 0, . . . , I, . . . , 0 Ψ−1 (θ) (Λ1 Hr + Λ2 B0 ) . (3.26) Remark 3.2.1. The matrix Ψ(θ) is a block matrix and in order to solve (3.22) one can use the Schur complement1 of the corresponding block matrix if it is invertible. This can reduce the required computational load of the problem. However, block matrices inside Ψ(θ) are not invertible. We can use the pseudoinverse of Ψ(θ) instead. In this case the inverse of Ψ(θ)T Ψ(θ) is required, for which the block matrices are invertible. Perturbation analysis The analysis in this section is based on the analysis techniques mentioned in [11] and [30]. Having the MPC solution at time step t, that is u(t, θ), our aim is to compute its derivatives with respect to θ, based on which the derivatives in (3.6) 1 The Schur complement of the block D of the matrix M = h A C i B is defined as A − BD−1 C. D 34 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC will be calculated. We start by considering a general observer of the form x̂(t + 1, θ|t) = f (x̂(t, θ|t), y(t, θ, θ̂N ), θ), x̂(t, θ|t) = g(x̂(t, θ|t − 1), y(t, θ, θ̂N ), θ), (3.27) where y(t, θ, θ̂N ) is the measured output, that is x(t + 1, θ, θ̂N ) = A(θ̂N )x(t, θ, θ̂N ) + B(θ̂N )u(t, θ), (3.28) y(t, θ, θ̂N ) = C(θ̂N )x(t, θ, θ̂N ) + e(t). We then arrive at formulating the following recursive procedure to compute the required derivatives at (3.6). Proposed method to compute derivatives of y with respect to θ 1. Initialization: set the initial values u∗ (0) = 0, x(0) = x̂(0|0) = x̂(0| − 1) = 0, du∗ (0) = 0, dx̂(0|−1) = 0, dx(0,θ) = 0; dθ dθ dθ 2. at a generic time t, start from the known values dx(t, θ) du∗ (t − 1, θ) dx̂(t, θ|t − 1) dy(t, θ, θ̂N ) , , , , dθ dθ dθ dθ to compute the following derivatives dx̂(t, θ|t) ∂g dx̂(t, θ|t − 1) ∂g dy(t, θ, θ̂N ) ∂g = + + , dθ ∂ x̂ dθ ∂y dθ ∂θ dLx (θ) dx̂(t, θ|t) du(t, θ) = x̂(t, θ|t) + Lx (θ) dθ dθ dθ dLu (θ) ∗ du∗ (t − 1, θ) + u (t − 1, θ) + Lu (θ) dθ dθ dLr (θ) + , dθ dx(t + 1, θ) dx(t, θ) du(u, θ) = A(θ̂N ) + B(θ̂N ) , dθ dθ dθ dy(t + 1, θ, θ̂N ) dx(t + 1, θ) = C(θ̂N ) ; dθ dθ (3.29a) (3.29b) (3.29c) (3.29d) (3.29e) (3.29f) 3. Continue by updating these values for the next step dx̂(t + 1, θ|t) ∂f dx̂(t, θ|t) ∂f dy(t, θ, θ̂N ) ∂f = + + dθ ∂ x̂ dθ ∂y dθ ∂θ du∗ (t, θ) du(t, θ) = . dθ dθ (3.30a) (3.30b) 3.3. NUMERICAL EXAMPLES 35 Finally, the proposed scheme to find an approximation of Vbapp is summarized in Algorithm 3.1. Algorithm 3.1 Proposed Method Initialization: choose Nu and M , set t = 1, x(0) = x̂(0| − 1) = 0, u(0) = 0, while t 6= M do solve P(t) for θ = θ̂N , Find active and inactive constraints and form I(t, θ̂N ), Solve (3.20) explicitly, Compute dy(t, θ, θ̂N ) employing (3.29a)-(3.29f), Update (3.30a)-(3.30b), end while 00 find V̂app (θ̂N ) using (3.6), The approximation of the cost function is then V̂app (θ) = 00 1 (θ − θ̂N )V̂app (θ̂N )(θ − θ̂N )T . 2 (3.31) Remark 3.2.2. The higher derivatives of the input and output signals can be computed simultaneously using the same procedure. Thus, a higher order Taylor expansions of the application function can be invoked to estimate the function. The method provides a fast tool for convex approximation of the application cost function. Many calculations in different time instants are the same and can be pre-computed. Moreover, the active constraints may not change often, thus, at each time instant a large number of the calculations can be skipped by re-using the results from previous time instants. Thus, the proposed approach is often faster than both the scenario-based and the ellipsoidal approximation method. 3.3 Numerical examples In this section we evaluate the proposed method in Section 3.2 with two numerical examples. Example 1 Consider the following system: x(t + 1) = θ2 x(t) + u(t), y(t) = θ1 x(t) + e(t). (3.32) The true system is given by the parameter values θ0 = [0.6 0.9]T and the measurement noise has the variance λ2e = 0.01. The objective is to find the application set 36 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC γ = 2000 1.1 1 1 θ2 θ2 γ = 1000 1.1 0.9 0.8 0.7 0.4 0.9 0.8 0.5 0.6 θ1 0.7 0.7 0.4 0.8 0.5 1.1 1 1 0.9 0.8 0.7 0.4 θ1 0.7 0.8 γ = 10000 1.1 θ2 θ2 γ = 5000 0.6 0.9 0.8 0.5 0.6 θ1 0.7 0.8 0.7 0.4 0.5 0.6 θ1 0.7 0.8 Figure 3.1: Level curves ( ) for the application set. The innermost curve corresponds to the ). The approximated app ( ) is also shown. The approximation is much required accuracy ( better for larger values of γ. The accepted scenarios, that is Vapp (θ) ≤ γ1 , for all cases are also shown (∗) . Θ, when MPC is used for reference tracking. We use the MPC formulation (3.3), with the following settings: Nu = Ny = 5, Q = 10, R = 1, umax = −umin = 1, ymax = −ymin = 2. We set the length of the experiment to N = 10 samples and the reference trajectory is a series of unit steps over the samples. Note that we use the application cost function defined in (3.4). Now using the proposed approach, we can obtain the application ellipsoid. 3.3. NUMERICAL EXAMPLES 37 The level curves for the application set together with the approximation of the set based on the proposed approach and the uniformly distributed scenarios are shown in Section 3.3. The results are studied for different values of accuracy, i.e γ. In order to check the accuracy of the proposed method, we perform a number of scenarios with different values of θ which are generated randomly with a uniform distribution. The experiment has been done for different values of γ. The results show that from 400 generated points for γ = 1000, 122 points are satisfying the condition Vapp (θ) < γ1 . Among all accepted values of θ, 90% are completely inside or on the border of the approximated ellipsoid, which means that the estimated ellipsoid covers at least 90% of the acceptable points. This value increases to 100% when γ = 10000. This mainly stems from the fact that the Taylor approximation of application cost function around θo is more accurate when we are closer to θo . Furthermore, the Hessian matrix is computed employing numerical methods, provided by DERIVESTsuite [13]. The application set is then approximated using the ellipsoidal approach (2.28). As expected, the result is the same as when the proposed method is used. However, in the proposed method, we need only one complete simulation of the closed loop system with MPC, while in the numerical approximation of the Hessian, which is based on finite difference approximation, O(6n2 ) simulations are required depending on the selected accuracy. Therefore, the new approach is expected to be faster. While it takes 94 seconds for the numerical method to calculate the Hessian matrix in this example, the new method needs only 12 seconds to give the same approximation, which means that 87% of time is saved. This example has only two unknown parameters and one state and thus the numerical approach is quite fast, however for more complicated examples the saved time will be more noticeable. Example 2 In this example we illustrate the algorithm on a distillation column simulation example. The nonlinear system representation is taken from a benchmark process proposed by the Autoprofit project [1]. For a general description of distillation columns, we refer the reader to [54]. The plant is linearised around the steady state operating conditions and then, using model order reduction methods, the second order model θ1 θ2 θ5 θ6 x(t + 1) = x(t) + u(t) θ3 θ4 θ7 θ8 (3.33) −0.8954 0.1421 y(t) = x(t) + e(t) −0.2118 −0.1360 is obtained, where e(t) is a white measurement noise with variance E{e(t)T e(t)} = 0.001. We assume that 1% performance degradation from the case when MPC is using the true parameters is allowed, that is, γ= 100 V (θ0 ) , 38 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC −4 Application cost function x 10 2 1 0 0 10 20 30 40 50 scenarios 60 70 80 90 100 Figure 3.2: Approximated (. - -) and real (o –) values of Vapp (θ) for 100 different samples of θ taken form a uniform distribution. where V (θ0 ) = 1 M PM t=1 ky(t, θ0 , θ0 ) − r(t)k2 , see [34]. Since MPC is used for tracking, the model is augmented with a constant output disturbance on each output to get integral action. This technique is presented in further detail in [43]. The proposed method has been employed to calculate the approximate application cost in (2.27). In order to evaluate the capability of the method, we run the process for 500 different values of θ, taken from a uniform distribution. Figure 3.2 shows the real and approximated values of the application cost function for 100 scenarios. In order to have a better insight, the samples which are located inside the application set are illustrated in Figure 3.3. It can be easily seen that the proposed method has a good performance inside the application set. Among 85 scenarios which result in an acceptable application cost, 83 scenarios are approximated as acceptable ones using the proposed method. The method classifies 6 points outside the region as acceptable ones. We consider two performance indexes to evaluate the results: false positive (FP) and false negative (FN). We introduce 1 if Vapp (θ) ≤ 1/γ 1(1/γ − Vapp (θ)) = , (3.34) 0 if Vapp (θ) > 1/γ and define Ni := {k such that 1(1/γ − Vapp (θ(k))) = i} for i = 0, 1. (3.35) 3.4. CONCLUSIONS 39 −4 1.4 x 10 Application cost function 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20 30 40 50 scenarios 60 70 80 Figure 3.3: Approximated (. - -) and real (o –) values of Vapp (θ) inside the application set. 92% of the samples inside the region are classified as acceptable ones by the proposed method. No. of scenarios 100 FP 0.4 FN 0.023 Table 3.1: Performance indexes achieved by the proposed method. Then FP(θ̂) FN(θ̂) := 1 |N0 | := 1 − P 1 |N1 | k∈N0 P ˆ (θ(k))), 1(1/γ − Vapp k∈N1 ˆ (θ(k))) 1(1/γ − Vapp (3.36) (3.37) ˆ FP captures the mistakes that for a generated θ, V app(θ) is estimated such that ˆ Vapp ≤ a/γ is satisfied while it is not the case for the true Vapp (θ). On the other ˆ (θ) > 1/γ while Vapp (θ) ≤ hand, FN counts the number of mistakes that the Vapp 1/γ. The results are summarized in Table 3.1. 3.4 Conclusions In this chapter we introduced a general technique for the approximation of the application set, a structure required for the implementation of optimal input design schemes. In particular, we have focused on MPC, a control technique for which it is not possible to obtain the application set explicitly. Some simulation examples 40 CHAPTER 3. APPLICATION SET APPROXIMATION FOR MPC have been presented, which show the advantages of the new method with respect to previous techniques, in terms of the necessary computations. The method is general enough to be applied to other controller strategies and application areas where it is not possible to derive the application set explicitly. In the next chapters this method will be used to estimate the application set for experiment design problem when there is no explicit control law for the controller. Chapter 4 Application Oriented Input Design: Time-Domain Approach Consider the general application oriented optimal input design problem (2.21). As mentioned in Section 2.1 of Chapter 2, for model structures linear in input, the Fisher information matrix is asymptotically an affine function of the input power spectrum. Therefore, the input design problem is usually formulated in the frequency domain and the outcome is an optimal input spectrum or an autocorrelation sequence, see (2.22). A realization of the optimal input signals are obtained from the given optimal spectrum, see [15]. In practice, however, there are often constraints on the input signals and the resulting output signals, which should be taken into account during the experiment design. These constraints are typically expressed in the time domain and how to handle them in frequency domain is not evident. One way to get around this problem is to impose these constraints during the generation of a time realization of the desired input spectrum, see [22], [40], [53] and [35]. In this chapter we introduce a new solution to the application oriented optimal input design. The objective is to satisfy the constraint (2.29) in minimum time while forcing the input and output signals to lie in certain convex sets. In the context of the problem (2.21), the experimental cost here is the minimum required time to satisfy the experiment constraint. The problem is formulated as a timedomain optimization problem, where it is straightforward to handle time-domain constraints on input and output signals by solving the problem directly in the time domain. 4.1 Problem formulation Consider the application oriented optimal input design problem (2.21). Assume that the experimental cost is the required time to satisfy the experiment design constraints while the experiment design constraints is relaxed using ellipsoidal approximation (2.29). To be able to formulate the optimal input design problem 41 42 CHAPTER 4. TIME-DOMAIN INPUT DESIGN mathematically, we introduce a slack variable to the experiment design constraint. The experiment design constraint (2.29) is then satisfied if a positive semidefinite matrix S exists such that: N Pθ−1 − χ2α (nθ )γ 00 Vapp (θo ) − S = 0, 2 S ≥ 0, (4.1) where S is the slack variable. We then try to minimize 2 χ2α (nθ )γ 00 , I (θ ) − J = V (θ ) − S app o F o 2 F (4.2) for S ≥ 0, under input and output constraints where k.kF denotes the Frobenius norm. Here we invoke (2.18) to replace N Pθ−1 with IF (θo ). The experiment design constraint is satisfied if J = 0. Remark 4.1.1. The Frobenius norm measures the Root Mean Square (RMS) gain of a matrix and it is easily calculated. However, one can use any other matrix norm. In order to find the minimum required time, we perform a time recursive input design algorithm. This also makes the algorithm compatible to be used with receding horizon controllers such as MPC. Hence, we formulate the input design problem as the following receding horizon problem, where at each time t we solve minimize t+Nu {u(k)}k=t ,S subject to 2 t+N χ2α (nθ )γ 00 u (θo ) − Jt = IF Vapp (θo ) − S 2 F S ≥ 0, u(k) ∈ U, k = t, . . . , t + Nu , y(k) ∈ Y, k = t, . . . , t + Ny , y(k) = G(q, θ0 )u(k), k = t, . . . , t + Ny . (4.3) Here, U and Y are convex constraint sets on the input and the output, respectively. These constraints on inputs and outputs could for instance correspond to saturation of actuators and the need to keep the system within a safe operating region. Nu and Ny are input and output horizons. If Ny is longer than Nu , the input can be considered zero or keep constant over the rest of the output horizon, see [43]. In this u chapter, we assume Nu = Ny for simplicity. It+N (θo ) is the Fisher information F matrix up to time t + Nu , an elaborate description is presented in Section 4.1. Although the solution to the problem (4.3) is a sequence of input values, we only apply the first value to the system and the optimization is performed again in the next time step, according to the receding horizon principle (see Section 2.4). At each time sample t, if the lower bound on the Fisher information matrix χ2 (n )γ 00 t+Nu is fulfilled, i.e. IF (θo ) ≥ α 2 θ Vapp (θo ), then Jt = 0 holds and vice versa. We can then stop running the receding horizon problem (4.3) if after applying the first value of the obtained input sequence Jt = 0 holds. We consider the time for 4.1. PROBLEM FORMULATION 43 which Jt = 0 hold for the first time as the minimum time required to satisfy the application requirements. To iteratively solve (4.3), we first need to rewrite the Fisher information matrix u It+N (θo ) in a recursive form and relate it to the input u(t). Then a cyclic algorithm F is proposed to address the input design problem (4.3). Remark 4.1.2. The problem could also be formulated as the dual problem of (2.22). This is achieved by gradually making the sets U and Y smaller until there is no input sequence that leads to Jt = 0 in finite time. As a result, we get the minimum experimental cost with which we can satisfy the experiment design constraint. This requires adding an outer loop to the problem (4.3) and solving (4.3) for different sets U and Y at each iteration of the loop. Remark 4.1.3. The formulation (4.3) can also be used to find the maximum accuracy γ, for which we can satisfy (2.29) in the sets U and Y. To this end, one can solve (2.29) for increasing values of γ until there is no input sequence that leads to Jt = 0 in finite time. This requires adding an outer loop to (4.3) and solving it for different values of γ at each iteration of the loop. Fisher information matrix For an unbiased estimator, the inverse of the Fisher information matrix is a lower bound on the covariance of the parameter estimation error, according to the CramérRao bound (2.16). The Fisher information matrix is ([21]): ( ) T ∂ log p(y N |θ) ∂ log p(y N |θ) IF (θ) := E ∈ Rnθ ×nθ , (4.4) ∂θ ∂θ where log p(y N |θ) is the log likelihood function. Considering the model (2.2) and assuming e(t) to be Gaussian white noise, the log likelihood function is: N log p(y N |θ) = constant − 1X T (t, θ)Λ−1 0 (t, θ) 2 t=1 (4.5) where N is the number of samples that are being used in the computation of the Fisher information matrix and (t, θ) ∈ Rny is the prediction error given by (t, θ) := H −1 (q, θ)[y(t) − G(q, θ)u(t)], (see Section 2.1 in Chapter 2). Assume that the plant and noise models are parameterized independently and let θG ∈ RnθG denote the parameters of the system model while θH ∈ Rnθ H contains the parameters of the noise model. The Fisher information matrix for data up to time t + Nu is T T ∂ (t,θ) ( ∂T (t,θ) ) k=t+N u X ∂θG −1 ∂θG t+Nu IF (θ) := E (4.6) Λ0 k=1 ∂T (t,θ) ∂θH ∂T (t,θ) ∂θH 44 CHAPTER 4. TIME-DOMAIN INPUT DESIGN T T where, ∂∂θ(t,θ) ∈ RnθG ×ny and ∂∂θ(t,θ) ∈ Rnθ H ×ny . Now, if we assume that {u(t)} G H and {e(t)} are uncorrelated (i.e. the system is operating in open loop), we obtain t+Nu ĪF (θG ) 0 t+Nu IF (θ) = E . (4.7) u 0 Īt+N (θH ) F Since ĪF (θH ) only depends on the noise e(t), the only part of the Fisher information matrix that can be optimized by the choice of the input signal is u Īt+N (θG ) F k=t+N Xu = k=1 ∂T (t, θ) ∂θG Λ−1 0 ∂T (t, θ) ∂θG T , (4.8) T u u is deterministic. One considering that E{Īt+N (θG )} = Īt+N (θG ), since ∂∂θ(t,θ) F F G can write T T k=t−1 X ∂T (t, θ) ∂ (t, θ) −1 u Λ (θ ) := Īt+N G 0 F ∂θG ∂θG k=1 (4.9) T T k=t+N X u ∂T (t, θ) ∂ (t, θ) −1 + Λ0 , ∂θG ∂θG k=t where the first term depends on the values of the input signal up to time t − 1, which are assumed to be known at time t. Therefore, we focus on the second term, which contains the inputs in the horizon in the optimization problem (4.3). Based on the definition of (t, θ) F1 (q)u(t) ∂T (t, θ) F2 (q)u(t) = (4.10) , .. ∂θG . FnθG (q)u(t) where h iT ∂G(q, θG ) u(t) . Fi (q)u(t) = − H −1 (q, θH ) ∂(θG (i)) (4.11) Building on [44], the elements of the reduced Fisher information matrix can be written as: u (Īt+N )i,j (θG ) F = (Īt−1 F )i,j (θG ) + k=t+N Xu (Fi (q)u(k))Λ−1 0 (Fj (q)u(k)), (4.12) k=t t−1 where i, j = 1, . . . , nθG and (ĪF )i,j (θG ) is obtained using the data available at time t. Denote the impulse response of Fi (q) by fi (t), the maximum length of the truncated impulse responses of Fi (q) for i = 1, . . . , nθG by n, and define (Ĩt−1 F )i,j (θG ) 4.1. PROBLEM FORMULATION 45 as the part of the Fisher information matrix depending on the future values of u(t). Define the matrix Fi ∈ R(Nu +1)ny ×(Nu +n)nu by (if Nu ≥ n): fi (n) fi (n − 1) 0 fi (n) Fi := . .. . . . 0 0 ... ... .. . ... ... ... ... ... .. . fi (n) . . . fi (1) fi (2) .. . 0 fi (1) .. . ... ... .. . fi (Nu + 1) ... ... 0 0 .. . , fi (1) (4.13) where ū∗ (t) := [u∗ (t − n + 1), . . . , u∗ (t − 1)] ū(t) ∈ R(n−1)nu , (Nu +1)nu ∈R := [u(t), . . . , u(t + Nu )] (4.14a) . (4.14b) The input values in (4.14a) are already known at time t while we are going to optimize the values in (4.14b). We can rewrite (4.12) as u (Ĩt+N )i,j (θG ) F = (ū∗ (t))T T ū(t) FiT Λ−1 e Fj ∗ ū (t) , ū(t) (4.15) −1 Λ−1 e = I(Nu +1)×(Nu +1) ⊗ Λ0 , see [44], which gives uT F1T Λ−1 e F1 u .. t+Nu (θG ) = ĨF . uT FnTθ Λ−1 e F1 u G ... .. . ... uT F1T Λ−1 e FnθG u .. , . T T −1 u Fnθ Λe FnθG u (4.16) G where T u = (ū∗ (t))T ū(t)T . (4.17) t+Nu t+Nu ĪF (θG ) = Īt−1 (θG ). F (θG ) + ĨF (4.18) Therefore, Defining −1 −1 Φ(u) := [Λe 2 F1 u, . . . , Λe 2 Fnθ u] ∈ R(Nu +1)ny ×nθG , (4.19) the Fisher information matrix can be written as: t+Nu ĪF (θG ) = Φ(u)T Φ(u) + Īt−1 F (θG ). (4.20) Since Φ(u) is linear in u, the Fisher information matrix is a quadratic function of the input sequence. 46 4.2 CHAPTER 4. TIME-DOMAIN INPUT DESIGN Optimization method: a cyclic algorithm For simplicity we assume that the application cost function depends only on the plant model. Thus we can use the reduced Fisher information matrix in (4.3). Substituting (4.20) into the cost function in (4.3) and with some abuse of notation (using θo instead of θG ) 2 Jt = Φ(u)T Φ(u) + C(t − 1) − S F , (4.21) where C(t − 1) = Īt−1 F (θo ) − χ2α (nθ )γ 00 Vapp (θo ), 2 (4.22) is a known matrix at time t which can be computed using the data available at time t. The optimization problem (4.3) is nonconvex and is in general hard to solve. However, the cost function is separable in terms of the variables, which makes it possible to find a solution of the problem through cyclic algorithms (see e.g. [60] and [59]). To put it another way, we can break the problem into two smaller problems by considering only one of the variables, u and S, at each time. The resulting problems are easier to solve. This motivates us to propose a cyclic algorithm for this problem. The method alternates between optimizing the cost function with respect to one of the variables while the other variable is fixed. The proposed algorithm consists of two main steps. Step 1 Assuming S is fixed to its most recent optimal value, Sopt , we aim to solve the following optimization problem at time instant t: Φ(u)T Φ(u) + C(t − 1) − Sopt 2 minimize F u ∗ T subject to u = (ū (t))T ū(t)T ∈ U, (4.23) u(k) = ū∗ (t), k = 1, . . . , n − 1, y(k) ∈ Y, k = t, . . . , t + Nu , y(k) = G(q, θ0 )u(k), k = t, . . . , t + Nu . where u(k) denotes the k th element of u. The optimization problem (4.23) is still nonconvex. However, the class of unconstrained signals, Φ(u), for which the cost function is zero, is ([59]) Φ(u) = U Sopt − C(t − 1) 21 , (4.24) if Sopt − C(t − 1) ≥ 0, (4.25) 4.2. OPTIMIZATION METHOD: A CYCLIC ALGORITHM 47 1 where U ∈ R(Nu +1)ny ×nθG is a semi-unitary matrix and (.) 2 is the Hermitian square root of a positive definite matrix. We will later show that the property (4.25) holds at time instant t − 1. Hence, the problem (4.23) can be relaxed to 1 2 minimize Φ(u) − U Sopt − C(t − 1) 2 u,U F T subject to u = (ū∗ (t))T ū(t)T ∈ U, u(k) = ū∗ (t), k = 1, . . . , n − 1, y(k) ∈ Y, k = t, . . . , t + Nu , y(k) = G(q, θ0 )u(k), k = t, . . . , t + Nu , (4.26) U T U = I. The cost function is still nonconvex. However, this problem, in turn can be broken into two sub-problems by considering only one of the variables and fixing the other one. Since Φ(u) is linear in u we formulate two convex problems: one in terms of u and another one in terms of U . Therefore, we can again use a cyclic optimization algorithm in order to solve the problem. Here, we will use the minimization algorithm suggested in [59]. The algorithm is alternating between the following two steps until convergence (one can define convergence when the tolerance of changes in the variables is smaller than a specified value): Step 1.1 : Assuming U is fixed to its most recent optimal value, solve the problem (4.26) for u, which is a constrained quadratic programming problem 1 2 uopt = arg min Φ(u) − Uopt Sopt − C(t − 1) 2 u F T subject to u = (ū∗ (t))T ū(t)T ∈ U, (4.27) u(k) = ū∗ (t), k = 1, . . . , n − 1, y(k) ∈ Y, k = t, . . . , t + Nu , y(k) = G(q, θ0 )u(k), k = t, . . . , t + Nu Step 1.2 : Having the optimal input sequence, uopt (t), find optimal U for (4.26) through a Singular Value Decomposition (SVD) 1 (4.28) (Sopt − C(t − 1) 2 Φ(uopt )T = Ū ΣŨ T , Uopt = Ũ Ū T . See [59] for more details. Step 2 Having obtained the optimal solution, uopt (t), from the first step, we need to solve Φ(uopt )T Φ(uopt ) + C(t − 1) − S 2 minimize F S (4.29) subject to S ≥ 0. 48 CHAPTER 4. TIME-DOMAIN INPUT DESIGN An important advantage of the proposed algorithm is that we can find a closedform solution for this step. The optimal solution of (4.29) is the projection of Φ(uopt )T Φ(uopt ) + C(t − 1) onto Sn+θ ([23]). To determine this projection, note that since Φ(uopt )T Φ(uopt ) + C(t − 1) is symmetric, we can write Φ(uopt )T Φ(uopt ) + C(t − 1) =V diag(λ1 , . . . , λnθ )V T , (4.30) where λi are the eigenvalues and V is the corresponding orthonormal matrix of eigenvectors. Thus Sopt = V diag max(0, λ1 ), . . . , max(0, λnθ ) V T . (4.31) See [23] for further information. Note that S ≥ Φ(uopt )T Φ(uopt )+C(t−1) according to (4.30) and (4.31), which confirms that the property (4.25) holds. The proposed method cycles between Step 1 and Step 2, the resulting problem involves solving a quadratic optimization problem, an SVD of a matrix with size nθ , and a projection. Therefore, the algorithm has sufficiently low complexity and thus it is fast enough to address large problems. Remark 4.2.1. We have no proof of convergence for the proposed method, yet. However, in the numerical simulations, good convergence results are obtained. We refer to [60] for more details and properties of alternating approaches and [59] for more examples The method is summarized in Algorithm 4.1. 4.3 FIR example To get a better insight into the proposed approach, we study it for a simple Finite Impulse Response (FIR) model y(t, θ) = θ1 u(t − 1) + θ2 u(t − 2) + e(t), E{e(t)} = 0 , E{e(t)2 } = λ, (4.32) where θ = [θ1 , θ2 ]. Assume that we aim to design an optimal input sequence with minimum length such that the identified model based on the obtained input signal can guarantee a desired control performance when it is being used in a controller. Moreover, we assume that because of some physical restrictions, we need |u(t)| ≤ umax , |y(t)| ≤ ymax . (4.33) In order to solve the problem we use the following steps. Desired control performance The desired performance can be specified by using the scheme defined in Section 2.2, where the application cost function is the same as (3.4), which can be replaced 4.3. FIR EXAMPLE 49 Algorithm 4.1 Proposed Cyclic Method Initialization: choose Nu and n Sopt ← 0, Uopt ← Uinit , t ← 1 and J0 6= 0 while Jt−1 6= 0 do i=1 while {Stopping criteria is not true} do Start Step 1: Solve (4.27) uiopt ← uopt Use uiopt and (4.28) to compute Uopt i Uopt ← Uopt Start Step 2: i and (4.30)-(4.31) to obtain Sopt Use uiopt , Uopt i Sopt ← Sopt i←i+1 end while u∗ (t) ←First sample of the optimal input signal Calculate ĪtF , C(t) and Jt t←t+1 end while t+N return optimal input sequence {u∗ (k)}k=1 u by its simulation-based approximation, (3.5), i.e. M 1 X b ky(t, θ̂N , θ̂N ) − y(t, θ, θ̂N )k2 , Vapp (θ) = M t=1 (4.34) The output signal is computed over a step response of the system with the controller running. We approximate the application set adopting the ellipsoidal approach presented in Section 2.2. The Hessian matrix can be calculated through either numerical ([13]) or analytical methods (see Chapter 3), depending on the type of controller and the number of parameters. Now, having defined Vapp (θ), we aim to design an input sequence such that (2.29) is fulfilled for a given γ. Input design The signal generation is done through the optimization problem (4.3). We first need to find the Fisher information matrix. Considering (4.32), we have (t) = y(t) − θ1 u(t − 1) − θ2 u(t − 2), −1 ∂T (t, θ) q u(t) = − −2 . q u(t) ∂θ (4.35) 50 CHAPTER 4. TIME-DOMAIN INPUT DESIGN Assume that we are at time instant t and that we aim to optimize the input signal over the prediction horizon of length Nu , putting n = 3, we can write (4.14a) and (4.19) as ū∗ (t) = [u∗ (t − 2), u∗ (t − 1)], ū(t) = [u(t), . . . , u(t + Nu )], u = [u∗ (t − 2), u∗ (t − 1), u(t), . . . , u(t + Nu )], 1 Φ(u) = √ [F1 u, F2 u], λ where F1 and F2 are obtained using (4.13). For example 0 −1 0 0 0 0 0 0 0 0 −1 0 0 −1 0 0 F1 = 0 0 0 0 0 0 −1 0 0 0 0 0 0 −1 −1 0 0 0 0 0 0 0 0 0 −1 0 0 −1 0 0 0 F2 = 0 0 0 0 −1 0 0 0 0 0 0 −1 0 (4.36) choosing Nu = 4, we have 0 0 0 , 0 0 (4.37) 0 0 0 , 0 0 and thus u (θ) = Ĩt+N F t+N 1 Xu u(k − 1)u(k − 1) u(k − 1)u(k − 2) . u(k − 2)u(k − 1) u(k − 2)u(k − 2) λ (4.38) k=t The Fisher information matrix for the FIR system is determined by the covariances of input sequences [58]. We are now ready to find the optimal input signal, ū(t), using the proposed alternating method. 4.4 Numerical results In this section we implement the suggested method on two examples. The first example is the FIR example explained in Section 4.3, whereas for the second example we consider an output error model with four unknown parameters. Example 1 Consider the FIR example in Section 4.3, with θo = [10, −9], umax = 0.5, ymax = 5, Nu = 5. Assume that we want to generate an input sequence of length N = 100 that, when used in an system identification experiment, satisfies both the application 4.4. NUMERICAL RESULTS 51 −7 θ2 −8 −9 −10 −11 8 9 10 θ1 11 12 Figure 4.1: Eapp is the outer ellipsoid and ESI is the inner one. ESI lies inside Eapp , which means the estimated parameters satisfy the application requirements with at least probability α. requirements and the input-output constraints. The identified model will be used in an MPC, with the cost function J= Ny X 2 ky(k + 1)k , (4.39) k=0 the same input and output constraints as during the experiment. We calculate the Hessian of the application cost function employing numerical methods, provided by the DERIVESTsuite [13]. The required accuracy is γ = 100 and we want that the estimated parameters lie in the identification set with probability α = 0.95. The method in Algorithm 4.1 is used to obtain an optimal input sequence. For the obtained input the slack variable S is strictly positive definite, and thus the experiment design constraint is satisfied. The application and identification ellipsoids for the obtained input are shown in Figure 4.1. The generated input signal is used in the system identification experiment with zero mean white Gaussian noise e(t) with variance λ = 1. One hundred θ̂N are estimated based on the measurements of y(t) for different realisations of e(t), when the obtained input signal is applied to the system. To this aim the system identification toolbox in Matlab is used. The results are shown in Figure 4.2. In total, 95% of the estimated parameters are inside the identification ellipsoid. It can be seen that ESI is inside Eapp , thus, the performance requirement will be fulfilled by the estimated parameters with probability 95%. The generated input signal is shown in Figure 4.3. It can be seen that the signal satisfies the constraints. However, it is worth noting that the constraints are only 52 CHAPTER 4. TIME-DOMAIN INPUT DESIGN −7 −7.5 −8 θ2 −8.5 −9 −9.5 −10 −10.5 −11 8 9 10 θ1 11 12 Figure 4.2: Eapp is the outer ellipse, ESI is the inner ellipse and θ̂N are the small circles. From 100 estimations, 95 estimations are inside the identification ellipsoid but the number of estimations inside the application ellipsoid is more than 95. applied on the noiseless output signal. Remark 4.4.1. In this simple example we now the true state since we are working with FIR models, thus we can impose constraints on the states directly. However, for more complicated model structures one needs to impose probabilistic bounds on the noisy output instead, see for example Chapter 5 of this thesis or [47]. We also formulate the problem in the frequency domain using (2.31) in Page 20, where the input power is chosen to be the experimental cost. To solve the problem, we use MOOSE1 . The result is shown in Figure 4.4. The obtained identification ellipsoids by the two approaches are match together. However, as mentioned before, the solution to (2.31) is an input spectrum and we need to find a corresponding time realization, which is not an easy problem under input and output constraints. An optimization based receding horizon approach has been proposed in [35]. Example 2 Consider the output error model of a two tank system: θ θ4 4.5 x(t + 1) = 3 x(t) + u(t), 1 0 0 y(t) = θ1 θ2 x(t) + e(t). 1A toolbox for optimal input design implemented in MATLAB [37]. (4.40) 4.4. NUMERICAL RESULTS 53 u(t) 0.5 0 −0.5 0 20 40 60 80 100 60 80 100 Time y(t) 5 0 −5 0 20 40 Time Figure 4.3: Generated optimal input and output signals. The constraints are satisfied. −7 θ2 −8 −9 −10 −11 8 9 10 θ1 11 12 Figure 4.4: Eapp is the outer ellipse. The identification ellipse obtained by MOOSE is shown by dotted lines (- -) and the one obtained by the proposed method is shown by solid line (–). An upper tank is connected to a pump with input u(t). The tank has a hole in the bottom with free flow into a lower tank, which also has a hole with free flow out of the tank. The level in the lower tank is the output, y(t). The true system parameters are given by [0.12 0.059 0.74 − 0.14]T . Assume that we aim to generate an input signal with length N = 100 such that the identified model satisfies the application requirements. The variance of the noise e(t) is λ = 0.01, and the 54 CHAPTER 4. TIME-DOMAIN INPUT DESIGN constraints are given by umax = 0.5, ymax = 5. Finally, the horizon is chosen as Nu = 5. The application cost function is chosen in the same way as Example 1, where the controller is an MPC with the cost function J= Ny X 2 ky(k + 1) − r(k + 1)kQ + k=0 Nu X 2 k∆u(k)kR , (4.41) k=1 where Q = I, R = 0.001I and r(t) is a step function. The proposed algo- u(t) 5 0 −5 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 Time 60 70 80 90 100 y(t) 5 0 −5 Figure 4.5: Input (top) and noiseless output (bottom), generated by the proposed method. rithm has been applied to the problem and the resulting input and output signals are shown in Figure 4.5. For the obtained input sequence, the constraint γχ2α (nθ ) 00 IF (θo ) > Vapp (θo ) is satisfied. This can also be confirmed by controlling 2 the eigenvalues of the slack variable Sopt , which are all positive and the zero cost function. 4.5 Conclusion In this chapter we introduced a new approach to application oriented optimal input design. The experimental cost is considered to be the time required for satisfying a lower bound on the Fisher information matrix. One significant feature of the proposed approach is that the problem is formulated in time domain and thus it is straightforward to handle constraints on the amplitude of the input and output signals. 4.5. CONCLUSION 55 The problem is, however, nonconvex. This is addressed using cyclic optimization methods, where we alternate between optimizing the cost function with respect to one variable while the others are fixed. We perform a time recursive algorithm and each optimization problem is solved in a receding horizon fashion. As a result, the method can also be used together with receding horizon controllers. The algorithm terminates when the application requirement is satisfied. We can converge to a local minimum with this sort of optimization algorithms. However, numerical examples showed that the method has good convergence properties and is consistent with previous results in the literature. The algorithm is also general enough to be applied to any linear system structure. Future research directions include extending the method to the closed-loop system identification and integrate it to Model Predictive Control and we aim to design optimal input while at the same time we are concerning about the control performance. More extensions could be robust and adaptive approaches. Chapter 5 Application Oriented Input Design for Closed-Loop Identification In Chapter 4 we studied the problem of application oriented optimal input design for open-loop systems. However, in practical applications, many systems can only work on closed-loop settings due to stability issues, production restrictions, economic considerations or inherent feedback mechanisms [12]. On the other hand, it is sometimes required to update the existing control laws or design a new controller. Since most of the methods for designing controllers require the knowledge of the system to be controlled, closed-loop system identification is a building block in this process. The main burden in closed-loop identification is the correlation between the measurement noise and input signal, which is imposed on the experiment by the feedback loop. In this case equation (4.7) in Chapter 4 does not hold any more and we need to find the correlation between the input signal and the output noise. The problem of closed-loop system identification is of great importance and has been investigated in the literature. There are three main approaches for closedloop identification: direct methods (the model is identified as if the system were in open-loop), indirect methods (the model is identified from the identified closedloop structure), and joint input-output (an augmented model is identified, where the input and output of the system are considered as the new outputs, and the reference and noise as new inputs); see e.g. [41, 55, 17, 65]. Since the input design problem plays an important role in any identification problem, closed-loop input design is also studied in the literature, which is often formulated as finding the spectrum of an additive external excitation for a fixed controller. There are a few works that design the controller at the same time. In this line, [25] performs the experiment design problem in frequency domain and the input spectrum is designed such that the uncertainty region for the identified model is minimized while satisfying constraints on the input power. The problem of closed-loop optimal input design for identification of linear time-invariant systems through PEM is studied in [24]. In this work the set of admissible controller-external 57 58 CHAPTER 5. CLOSED-LOOP INPUT DESIGN input pairs is parametrized by a finite set of matrix valued trigonometric moments. In [31, 29] the problem of experiment design for closed loop system identification is analyzed. In this case, [31, 29] use a finite dimensional parameterization of the input spectrum and the Youla-Kucera parameterization to recast the problem as a semidefinite program. However, the existing methods have some restrictions in the case of nonlinear feedback and constraints on the input and output. In this chapter we focus on extension of application oriented experiment design for closed-loop system identification under input and output constraints. We consider a general controller (either linear or nonlinear) with an explicit control law, where the main goal is reference tracking under probabilistic bounds on input and output signals. Due to a performance degradation (e.g., a change in the process dynamics producing a degradation in the quality of reference tracking), we want to update the current controller or design another one, and thus a plant model needs to be identified. We then design an experiment to collect data for identification. Since the controller is known we perfor indirect system identification. This is done by adding an external stationary input. The problem is thus formulated as an optimization, where we design the external excitation achieving the minimum experimental effort. Meanwhile, we are also taking care of the tracking performance of the existing controller. We add the experiment design constraint (equation (2.29), Page 19), to retrieve an exciting enough input signal. The last guarantees that the estimated model is in the set of models that satisfies the desired control specifications, with a given probability. In practice we also have bounds on the input and output signals, which should be taken into account during the experiment design. Thus, the optimization also considers bounds for the input and output of the system. Note that the measured output is usually corrupted by measurement noise, which is described by a stochastic model. Therefore, it is impossible to impose deterministic bounds on the output. This is the same for the input signal since we are working in the closed-loop. Consequently, in the proposed optimization problem probabilistic bounds are considered for the input and output of the system. The obtained optimization problem is nonconvex due to the constraints, and thus it is difficult to handle. This issue is relaxed by extending the method introduced in [62, 63] for closed-loop and constrained system identification, where it is assumed that the external excitation is a realization from a stationary with finite memory and finite alphabet. The probability distribution function associated with the external excitation is thus characterized as a convex combination of basis inputs, which are known. The method allows us to use Monte-Carlo methods to approximate the cost functions, probabilities and information matrices associated with each basis input. The resulting problem is convex on the decision variables, which makes it tractable. In the rest of this chapter the problem of application oriented experiment design for closed-loop system identification is defined. We then present the proposed algorithm to find a convex approximation of the problem. We finally evaluate the proposed algorithm throughout a numerical example. 5.1. PROBLEM DEFINITION 59 et noise model rt ut + − plant ++ controller yt +− yd Figure 5.1: The schematic representation of the closed-loop system 5.1 Problem definition Let {A(θo ), B(θo ), C(θo )} be an observable state space realization of the transfer function G(q, θo ) in (2.1), Page 11. The system (2.1) can thus be rewritten as: x(t + 1) = A(θo )x(t) + B(θo )u(t), y(t) = C(θo )x(t) + ν(t), (5.1) where u(t) ∈ Rnu and y(t) ∈ Rny are the input and output vectors. ν(t) ∈ Rne is a coloured noise with ν(t) = H(q; θo )e(t), where H is a rational noise filter in terms of the time shift operator q, and {et } is white noise sequence with zero mean and covariance matrix Λe . In addition, we assume that H is stable, inversely stable, and satisfies H(∞; θo ) = I. Assume that the system (5.1) is controlled, using a general (either linear or nonlinear) output feedback controller: u(t) = r(t) − Ky,t (y t ), (5.2) where Ky,t is a θ-independent, known and time varying function, r(t) denotes an external excitation, and y t := {yk − yd }tk=1 . The feedback (5.2) is such that the output signal tracks a desired reference value yd . For simplicity, we assume that yd is constant. However, we can also consider a time varying value, yd (t), provided that it is a stationary process. The closed-loop structure is shown in Figure 5.1. Suppose that we aim to find a model of system (5.1) with the same structure which is parameterized by an unknown parameter vector θ ∈ Rnθ , that is: x(t + 1) = A(θ)x(t) + B(θ)u(t), y(t) = C(θ)x(t) + ν(t). (5.3) Thus the closed-loop model will be x(t + 1) = Ft (θ, x(t), y t ) + B(θ)r(t), y(t) = C(θ)x(t) + ν(t), (5.4) 60 CHAPTER 5. CLOSED-LOOP INPUT DESIGN where ν(t) = H(q; θ)e(t), and Ft (θ, x(t), y t ) := A(θ)x(t) − B(θ)Ky,t (y t ). We assume that the resulting closed-loop system (5.4) is asymptotically stable. Remark 5.1.1. If (5.4) is asymptotically stable, and the feedback controller is linear, then the information regarding closed-loop stability can be included in the problem formulation by using the Youla parametrization. Since we know that the closedloop is stable, then we can parametrize the model structure for the system in the set of linear time invariant models that can be stabilized by the given controller, which is performed by using the Youla parameter [64]. The objective is to design an experiment for the closed-loop system (5.4), that generates M samples of the external excitation r(t), to be used for identification of the unknown parameters θ in (5.3). To this end, we consider the experiment design problem (2.29). Since the system is in the closed-loop we need to keep the output of the plant y(t) close to yd during the identification experiment. Hence, we choose to minimize the following experimental cost in the optimal input design problem (2.21) (M ) X 2 2 J =E ky(t) − yd kQ + k∆u(t)kR , (5.5) t=1 where ∆u(t) = u(t) − u(t − 1), (5.6) and Q and R are positive definite matrices. The first term in (5.5) penalizes the deviations from the desired output, while the second term is responsible for minimizing the input energy. The expected value is with respect to {r(t)} and {e(t)}. In practical applications, it is common to have bounds on the maximal input and output amplitudes allowed by the process. These constraints appear due to physical limitations and/or to preserve the system in a safe operating point. However, since both the input and output of the system contains a stochastic process which cannot be measured, these bounds cannot be forced in deterministic sense. Therefore, the input and output constraints are quantified by using a probability measure. Thus, we consider the following probabilistic constraints during the identification process1 : P{|y(t) − yd | ≤ ymax } > 1 − y , t = 1, . . . , M, P{|u(t)| ≤ umax } > 1 − x , t = 1, . . . , M, (5.7) where umax is the maximum allowed value for the input signal and ymax is the maximum allowed deviation of the output from its desired value, based on the physical properties of the system and actuators; and x with y are two design variables that define the desired probability of being in the safe bounds for the input and output signals. 1 The inequalities in (5.7) are element-wise operations. 5.2. CONVEX APPROXIMATION OF THE OPTIMIZATION PROBLEM VIA GRAPH THEORY 61 In addition to the previous constraints, we require that the updated (or newly) designed controller based on the estimated parameters can guarantee an acceptable control performance, i.e. the experiment design constraint (2.29) is satisfied. The optimization problem can be summarized as: Problem 5.1. Design {r(t)opt }M t=1 as the solution of minimize {r(t)}M t=1 J =E (M X ) ky(t) − 2 yd k Q + 2 k∆u(t)kR , t=1 subject to x(t + 1) = Ft (θ, x(t), y t ) + B(θ)r(t), y(t) = C(θ)x(t) + ν(t), ν(t) = H(q; θ)e(t), t u(t) = r(t) − Ky,t (y ), P{|y(t) − yd | ≤ ymax } > 1 − y , P{|u(t)| ≤ umax } > 1 − x , IF (θ) ≥ t = 1, . . . , M, t = 1, . . . , M, (5.8) t = 1, . . . , M, t = 1, . . . , M, t = 0, . . . , M − 1, γχ2α (n) 00 Vapp (θ), 2 where IF (θ) is the Fisher information matrix obtained with M samples. Note that Problem 5.1, has a very similar structure to MPC, see [43]. However, they are not necessarily the same since we are not considering a receding horizon approach in this problem. The optimization problem (5.8) is nonconvex due to the possible nonlinearity of the closed-loop system and the experiment design constraints and is difficult to solve explicitly. Moreover, computing the probability measures and expected value are also challenging problems. These issues are relaxed by extending the method introduced in [62, 63] for closed-loop and constrained system identification. In the next section we explain how we can find a convex approximation of the problem using the idea of input design via graph theory. Remark 5.1.2. Problem 5.1 relies on the knowledge of the true system. This can be addressed by either implementing a robust experiment design scheme on top of it [52]; or through an adaptive procedure, where the Hessian of the cost function and output predictions are updated as more information is being collected, [19]. In the rest of this chapter we rely on the knowledge of θo (or a prior estimate of it). 5.2 Convex approximation of the optimization problem via graph theory To find a convex approximation of Problem 5.1 we start by making the following assumptions on the external excitation {r(t)}M t=1 : 62 CHAPTER 5. CLOSED-LOOP INPUT DESIGN Assumption 5.2.1. {r(t)}M t=1 is a realization of a stationary process, {r(t)}. Assumption 5.2.2. The stationary process {r(t)} has a finite memory length, nm such that M > nm . By the Assumptions 5.2.1 and 5.2.2 we restrict {r(t)}M t=1 to those stationary processes that can be retrieved as an extension of probability density function (pdf) with a finite number of elements. Thus, Problem 5.1 can be formulated as finding the optimal pdf for the external excitation signal, instead of the signal {r(t)}M t=1 . It is then enough to characterize the set of corresponding pdf’s for {r(t)}M t=1 under the aforementioned assumptions. To proceed we assume the following: Assumption 5.2.3. The signal r(t) belongs to the set C, which has finite cardinality, denote by nC . We also introduce: Definition 5.2.1. (Probability mass function) A probability measure whose support is a set with finite cardinality is called probability mass function (pmf) and denoted by p. Under the Assumption 5.2.3 and Definition 5.2.1, any pmf of {r(t)}M t=1 , p : C → R, satisfies the following properties: nm p(x) ≥ 0 for all x ∈ C nm , P x∈C nm p(x) = 1, P P (nm −1) , v∈C p(v, z) = v∈C p(z, v), for all z ∈ C (5.9a) (5.9b) (5.9c) where Equations (5.9a)-(5.9b) are hold since the function p is a pmf and (5.9c), which is called shift invariant property, is necessary to make p a valid pmf for a stationary signal. We can then characterize the set of pmfs of {r(t)}M t=1 as n p(x)P≥ 0 for all x ∈ C m , p(x) = 1, PC = p : C nm → R P . (5.10) n Px∈C m (nm −1) p(v, z) = p(z, v), for all z ∈ C . v∈C v∈C The arising question is how we can parametrize the obtained set? This question is answered in the next subsection. Parametrization of PC We start by noticing that the set PC is described by finite number of linear equalities and inequalities and thus it is a polyhedron [50, pp. 170]. Hence, any element in PC can be written as a convex combination of its extreme points, see [50, Corollaries 18.3.1 and 19.1.1], that is for each p ∈ PC p= nv X j=1 βj pj , (5.11) 5.2. CONVEX APPROXIMATION OF THE OPTIMIZATION PROBLEM (u t 1 , ut ) (u t 1 , ut ) (0 (0 , 0) 63 , 1) (u t 1 , ut ) (u t 1 , ut ) (1 , 0) (1 , 1) Figure 5.2: The graph derived from C = {0, 1} and nm = 2. Each node represents one of the possible states in C 2 , and each directed edge shows the possibility of mving from one node to the other one. The prime cycles are shown in red. These cycles are in one-to-one correspondence with the extreme points of PC . where pj ’s are the extreme points of PC and nv is the number of extreme points of PC and nv X βj = 1, βj ≥ 0. (5.12) j=1 Denote the set of extreme points of PC by VPC := {pj ; j = 1, . . . , nv }, it is then adequate to characterize VPC in order to find PC . In this thesis we invoke the proposed approach in [62, 63] to find the extreme points of PC . The underlying idea is to rely on the connection between stochastic processes and graph theory. However, we are not going into detail of this approach since it is beyond the scope of this thesis. The general idea is explained by a simple example. Example 5.2.1 (Find extreme points via graph theory). Assume that the sequence {r(t)} is a realization form a stationary process with a finite memory length nm = 2 and the signal r(t) belong to the set C = {0, 1}, which has finite cardinality. The set of pmfs of r(t) is then a polyhedron defined by (5.10). Thus any element inside the set can be written as a convex combination of its extreme points. In order to find these extreme points by graph theory we start by finding all the possible states in C nm and associate each state to a node in a directed graph, see Fig- 64 CHAPTER 5. CLOSED-LOOP INPUT DESIGN (uk−1 , uk ) (0,0) (0,1) (1,0) (1,1) (uk , uk+1 ) {(0,0), (0,1) {(1,0), (1,1) {(0,0), (0,1) {(1,0), (1,1) } } } } Table 5.1: Possible transitions from (uk−1 , uk ) to (uk , uk+1 ). ure 5.2. We can then compute the possible transition from the state (uk−1 , uk ) to (uk , uk+1 ) which is corresponding to one directed edge in the graph. It is shown in [63, Theorem 2.1] that each pj ∈ VPC corresponds to a uniform distribution whose support is the set of elements of a prime cycle in the generated graph. A prime cycle is a closed path in which no node except the first and the last appears twice and does not accommodate another closed path as its subset, see [63] for more elaborated description. Consider the prime cycle {(0, 1), (1, 0), (0, 1)} in Figure 5.2. A realization associated with this prime cycle is obtained by taking the last element of each node, that is N −1 {r(t)}N + 1 /2}. (5.13) t=1 = {1, 0, 1, 0, . . . , (−1) Once all the extreme points, pj , are computed, the set VPC is fully determined. Consider again the Problem 5.1, we aim to formulate this problem in terms of pmf M of {r(t)}M t=1 . Moreover, based on (5.11), any pmf of {r(t)}t=1 can be written as a convex combination of some known extreme points. Thus, in order to find the optimal pmf of {r(t)}M t=1 as a solution of the Problem 5.1, we need to first evaluate the cost function and constraints for each extreme point. We can then rewrite Problem 5.1 in terms of the weights, βj ’s, and find the optimal pmf for the external excitation by computing the optimal weights, βj ’s. Cost function v Since the set {pj }nj=1 is known (each pj is a uniform distribution over the nodes in the j-th prime cycle, see Example 5.2.1), we can sample {r(t)(j) }N t=1 from each pj (with N sufficiently large). We can then approximate the expected value Ee(t),r(t)(j) {·} associated with the extreme point pj by Monte-Carlo simulations as N 2 2 2 2 1 X (j) (j) Ee(t),r(t)(j) y(t) − yd + ∆u(t) ≈ y(t)(j) − yd +∆u(t)(j) . N t=1 Q Q R R (5.14) The approximated cost function in terms of the weights is J= nv M X X t=1 j=1 βj Ee(t),r(t)(j) 2 2 . y(t)(j) − yd + ∆u(t)(j) Q R (5.15) 5.2. CONVEX APPROXIMATION OF THE OPTIMIZATION PROBLEM 65 The obtained cost function is convex in terms of βj ’s. Amplitude constraints The probability measures can be approximated for each extreme point, pj , by Monte-Carlo simulations as Pe(t),r(t)(j) {|y(t)(j) − yd | ≤ ymax } ≈ N 1 X , 1 (j) N t=1 |y(t) |≤ymax Pe(t),r(t)(j) {|u(t)(j) | ≤ umax } ≈ N 1 X 1 , (j) N t=1 |u(t) |≤umax where 1X = 1 if X is true, and 0 otherwise. The constraints can then be expressed in terms of the weights, that is nv X βj Pe(t),r(t)(j) {|u(t)(j) | ≤ umax } > 1 − x , j=1 nv X (5.16) (j) βj Pe(t),r(t)(j) {|y(t) − yd | ≤ ymax } > 1 − y , j=1 which are convex for the weights, βj ’s. Experiment design constraint The computational of the Fisher information matrix in the experiment design constraint is based on the presented method in [63], where numerical simulations are employed to compute the Fisher information matrices associated with each mea(j) (j) v sure in the set {pj }nj=1 , IF (θ). The computation of IF (θ) is analyzed in the next subsection. The experiment design constraint can be approximated by nv X j=1 (j) βj IF (θ) ≥ γχ2α (n) 00 Vapp (θ). 2M (5.17) We notice that the Fisher information matrix is expressed as convex combination (j) v of the respective quantities achieved by the external excitations {r1:M }nj=1 and thus the experiment design constraint is LMI, even for nonlinear systems. Finally, the optimization problem can be expressed in terms of the weights of the extreme points. The obtained problem is convex and can be written as 66 CHAPTER 5. CLOSED-LOOP INPUT DESIGN Problem 5.2. Find the optimal weights {β1 , . . . , βnv } as a solution of minimize {β1 , ..., βnv } subject to J= nv M X X o n 2 2 βj Ee(t),r(t)(j) y(t) − y d Q + k∆u(t)kR , t=1 j=1 x(t + 1)(j) = Ft (θ, x(t)(j) , y t (j) ) + B(θ)r(t)(j) , y(t)(j) = C(θ)x(j) (t) + ν(t), t = 1, . . . , M, ν(t) = H(q; θ)e(t), t = 1, . . . , M, (j) u(t)(j) = r(t)(j) − Ky,t (y t ), t = 1, . . . , M, nv X βj Pe(t),r(t)(j) {|u(t)(j) | ≤ umax } > 1 − x , j=1 nv X (5.18) βj Pe(t),r(t)(j) {|y(t)(j) − yd | ≤ ymax } > 1 − y , j=1 nv X (j) βj IF (θ) ≥ j=1 nv X γχ2α (n) 00 Vapp (θ), 2M βj = 1, βj ≥ 0, j = 1, . . . , nv . j=1 (j) (j) (j) (j) (j) In the Problem 5.2, r1:M , u1:M , x1:M , y1:M , and y t denote the signals ob(j) tained when r1:M is drawn from pj , j ∈ {1, . . . , nv }. In problem (5.18), we aim v to minimize the cost J by finding the normalized weights {βj }nj=1 satisfying the probabilistic bounds on the input and output, and the requirement on the accuracy of the parameter estimates. The first four equations in the constraints of (5.18) are the equations given by the closed loop system, which are defined for the external (j) nv excitations {r1:M }j=1 . The rest, represent the probabilistic bounds for the input and the output, the constraint on the accuracy of the parameter estimates, and the v requirement that {βj }nj=1 are nonnegative, normalized weights. 5.3 Fisher information matrix computation To integrate the experiment design constraint with the optimization problem (5.18), (j) we need to compute the Fisher information matrix IF (θ) for each {r(t)j }M t=1 associated with the j-th extreme point of the set of stationary processes of memory nm and alphabet C. We recall for an unbiased estimator, the inverse of the Fisher information matrix is a lower bound on the covariance of the parameter estimation error, according to the Cramér-Rao bound. The Fisher information matrix is given by ((2.17) in 5.3. FISHER INFORMATION MATRIX COMPUTATION 67 Page 15) n ∂ log p(y M |θ) ∂ log p(y M |θ) o IF (θ) := E ∂θ ∂θT ∈ Rnθ ×nθ , (5.19) θ=θo where y M := {y1 , . . . , yM }. We notice that (5.19) is equal to n ∂ 2 log p(y M |θ) o ∈ Rnθ ×nθ . IF (θ) = −E ∂θ∂θT (5.20) θ=θo Due to the randomness of e(t), (5.4) can be rewritten as x(t + 1) ∼ fθ (x(t + 1)|y t , x(t), r(t)) , yt ∼ gθ (y(t)|x(t)) , (5.21) where fθ (x(t + 1)|x(t), r(t)) and gθ (y(t)|x(t)) denote the pdf of the state x(t + 1), and output y(t), conditioned on the knowledge of {x(t), r(t)} and y M . Using the model description (5.21), and the Markov property of the system (5.4), we can write the log likelihood of the joint distribution of y M and xM {x(t)}M t=1 as log pθ (xM , y M ) = M −1 X log{fθ (x(t + 1)|y t , x(t), r(t))} t=1 + M X (5.22) log{gθ (y(t)|x(t))} + log{pθ (x(1))} . t=1 To simplify the analysis, we will assume that the distribution of the initial state is independent of θ, i.e., pθ (x(1)) = p(x(1)). To compute the Fisher information matrix we consider two cases: the feedback is linear; the feedback in nonlinear. In the first case we can compute the information matrix in the frequency domain while for the second case numerical methods can be used to find an approximation of the Fisher information matrix. We explain each case in more details. Linear feedback When Ky,t is a linear controller, expressions (5.19) and (5.20) can be computed in the frequency domain [41, Section 9.4]. Since we know {r(t)(j) } for each j ∈ (j) {1, . . . , nv }, it is possible to compute its corresponding spectrum, say Φr (ω). To (j) this end, we notice that {r(t) } is a periodic sequence with period given by Tj . (j) Using [41, Example 2.3] we compute Φr (ω) as Φ(j) r (ω) = Tj −1 2π X j, p Φr (2πk/Tj )δ(ω − 2πk/Tj ) , 0 ≤ ω < 2π , Tj k=0 (5.23) 68 CHAPTER 5. CLOSED-LOOP INPUT DESIGN where δ is the Dirac delta function, and Tj −1 p Φj, r (ω) := X Rr(j) (τ )eiωτ , (5.24) Tj 1 X (j) (j) T r rt−τ . Tj t=1 t (5.25) τ =0 Rr(j) (τ ) := Remark 5.3.1. Considering (5.24), we can conclude that in the case of linear feedback we will have a multisine excitation. Nonlinear feedeback On the other hand, when Ky,t is a nonlinear function, equations (5.19) and (5.20) often result in complex (and almost intractable) expressions. Thus, we will approximate the Fisher information matrix using numerical methods, instead. One solution is to use particle methods to approximate (5.19) as the covariance maM |θ) trix of the gradient of the log-likelihood function, ∂ log p(y (score function) [61]. ∂θ Another approach is based on the numerical computation of (5.20) using small perturbation methods [57], where the Hessian (5.20) is computed as an average of numerical approximations based on the score function. An elaborated description of the small perturbation method is provided here. Fisher information matrix approximation through the small perturbation method Consider the Fisher information matrix given by (5.20). Our goal is to obtain an estimate of (5.20) using the small perturbation method. The main advantages of the small perturbation method is that it can approximate (5.20) even if the gradient of the function is not available and the method can be implemented simply. We described the technique in [57] to approximate (5.20) and the materials are taken from [63]. The main idea in [57] is to compute the Hessian (5.20) based on synthetic data, i.e., realizations of {y(t)}, and {u(t)} are obtained by simulation of the model (5.4) for small perturbations of θo , using realizations of {r(t)} and {e(t)}. We (i) will denote the i-th synthetic data as Z N = {y(t)(i) , u(t)(i) , r(t)(i) , e(t)(i) }N t=1 , i ∈ {1, . . . , N }. Before we continue, we define ∆(k, i) ∈ Rnθ as a zero mean random vector such that its entries are independent, identically distributed, symmetrically distributed random variables that are uniformly bounded and satisfy n o −1 E ∆(k, i), l < ∞, with ∆(k, i), l denoting the l-th entry of ∆(k, i) . We notice that the latter condition excludes uniform and Gaussian distributions. Furthermore, we assume that ∆(k, i), l are bounded in magnitude. A valid choice is the Bernoulli distribution, defined on {−1, 1}, and with parameter η = 0.5. 5.4. NUMERICAL EXAMPLE 69 (i) On the other hand, for each realization Z N , we compute the k-th approximation of (5.20) (k ∈ {1, . . . , M }) as ( ) δG> 1 δG(k, i) −1 > (k, i) (k, i) −1 ∆(k, i) , (5.26) + ∆(k, i) ÎF := − 2 2c 2c where, (i) δG(k, i) := ∇θ log pθ (y1:N ) θ=θo +c∆(k, i) (i) − ∇θ log pθ (y1:N ) θ=θo −c∆(k, i) , (5.27) (i) (i) N ∆−1 (k, i) denotes the entry-wise inverse vector of ∆(k, i) , y1:N = {y(t) }t=1 , and c > 0 is a predefined constant. In [56] it is shown that, under suitable assumptions, the bias of the Fisher information matrix estimate (5.26) is of order O(c2 ). The main assumption in [56] is the smoothness of log pθ (y1:N ), which is reflected in the assumption that ∇θ log pθ (y1:N ) is thrice continuously differentiable function in a neighbourhood of θo . However, the main limitation of (5.26) is that it is a low rank (k, i) approximation of the Fisher information matrix (ÎF has at most rank two), and it may not even be positive semi-definite. In [57] an improved estimate of the Fisher information matrix is provided as a function of (5.26). To this end, we define D(k, i) ∈ Rnθ ×nθ as > −I, D(k, i) := ∆(k, i) ∆−1 (k, i) (5.28) and the function Ψ(k, i) : Rnθ ×nθ → Rnθ ×nθ as Ψ(k, i) (H) := 1 1 > HD(k, i) + D(k, i) H . 2 2 (5.29) Based on (5.28) and (5.29), we can write the new Fisher information matrix estimate in recursive form (in i) as (M, i) IF = M (M, i−1) i i − 1 (M, i−1) 1 X h (k, i) IF + ÎF + Ψ(k, i) I F , i iM (5.30) k=1 (M, 0) where I F = 0. In [57] is shown that the estimate (5.30) tends to IF + O(c2 ) as N → ∞ (for a fixed M ) in mean square sense. The advantage of using the estimator (5.30) over (5.26) is that (5.30) guarantees that the estimate will be a full rank and positive-definite matrix. 5.4 Numerical example To illustrate the previous discussion, we introduce the following example: 70 CHAPTER 5. CLOSED-LOOP INPUT DESIGN rt 0.5 0 −0.5 0 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 rt 0.5 0 −0.5 0 rt 0.5 0 −0.5 0 t Figure 5.3: Part of the external excitation {r(t)}500 t=1 . Top: Optimal external excitation. Middle: Optimal external excitation without probabilistic constraints. Bottom: Random binary signal. Example 1 Consider the open-loop, SISO state space system described by x(t + 1) = θ20 x(t) + u(t) , (5.31a) θ10 y(t) = x(t) + e(t) , (5.31b) 0 T T with true parameters θ0 := θ1 θ20 = 0.6 0.9 . The system is controlled in closed-loop using the controller u(t) = r(t) − Ky y(t) , (5.32) where Ky = 0.5 is a known constant. The objective is to identify the open-loop T T parameters θ := θ1 θ2 from the identified closed-loop ones θc := θ1c θ2c in the model x(t + 1) = θ2c x(t) + r(t) − Ky e(t) , y(t) = θ1c x(t) + e(t) , (5.33a) (5.33b) 5.4. NUMERICAL EXAMPLE 71 ut 1 −1 0 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 ut 1 −1 0 ut 1 −1 0 t Figure 5.4: Part of the input signal {u(t)}500 t=1 . Top: Input signal for the optimal reference. Middle: Input signal for the optimal reference without probabilistic constraints. Bottom: Input signal for a random binary reference. using the transformation law θ1 = θ1c , θ2 = θ1c + Ky,t θ1c . (5.34a) (5.34b) To this end, we design the external excitation {r(t)}500 t=1 as a realization of a stationary process with memory nm = 2, and subject to r(t) ∈ C := {−0.5, −0.25, 0, 0.25, 0.5}, for all t ∈ {1, . . . , 500}. Since the experiment will be performed in closed-loop, we define the following cost function to measure performance degradation 500 Vapp (θ) := 1 X ky(t, θo ) − y(t, θ)k22 , 500 t=1 (5.35) where y(t, θ) denotes the closed-loop output when θ is employed to describe the open loop model and a linear output feedback controller with constant gain ky has been used. Finally, we will solve the approximate problem (5.18), where yd = 0, for all t ∈ {1, . . . , 500}, Q = 1, R = 0.02, εy = εx = 0.07, ymax = 2, umax = 1, γ = 102 , and α = 0.98. 72 CHAPTER 5. CLOSED-LOOP INPUT DESIGN yt 2 0 −2 0 20 40 60 80 100 20 40 60 80 100 20 40 60 80 100 yt 2 0 −2 0 yt 2 0 −2 0 t Figure 5.5: Part of the output {y(t)}500 t=1 . Top: Output signal for the optimal reference. Middle: Output signal for the optimal reference without probabilistic constraints. Bottom: Output signal for a random binary reference. Figure 5.3 presents one realization of {r(t)}500 t=1 obtained by solving (5.18), one realization of {r(t)}500 obtained by solving (5.18) without probabilistic constraints, t=1 and from a random binary sequence with values {−0.5, 0.5}. From this figure we see that the optimal sequence is zero most of the time, except for short pulses. This can be explained from the tight probabilistic bounds imposed for {u(t)}, which restricts the excitation provided by {r(t)}. If we compare the previous signal with the one obtained by solving (5.18) without probabilistic bounds, we see that the external excitation contains more oscillations when the probabilistic bounds are removed. Figures 5.4 and 5.5 present one realization for the resulting input {u(t)}500 t=1 and output {y(t)}500 , respectively. From those realizations, we conclude that, for the t=1 optimal reference, the input and output are inside the limiting regions 93.8% and 96% of the time, respectively, which satisfies the design requirements. On the other hand, for the external excitation obtained by solving (5.18) without probabilistic bounds, we have that the input and output satisfies the constraints 86.6% and 93.4% of the time, respectively. Therefore, in this example we need to incorporate the probabilistic bounds to guarantee that both the input and output of the system 5.4. NUMERICAL EXAMPLE 73 1 1 θ2 1.1 θ2 1.1 0.9 0.9 0.8 0.8 0.7 0.7 0.4 0.6 θ1 0.8 0.4 0.6 θ1 0.8 Figure 5.6: Application ellipsoid (green, dot-dashed line) with the respective identification ellipsoids. Blue, continuous line: Identification ellipsoid for the random binary reference (realizations marked with ∗). Red, continuous line: Identification ellipsoid for the optimal reference with probabilistic bounds (realizations marked with circles). Black, dashed line: Identification ellipsoid for the optimal reference without probabilistic bounds (realizations marked with triangles). are inside the desired region with the prescribed confidence level. With the previous modification, we restrict the set of optimal feasible solutions for the problem of minimum variance to the subset of optimal solutions satisfying the probabilistic bounds. Finally, for the random binary reference, the input and output are inside the confidence region 90.8% and 79.6% of the time, this fact does not satisfy the confidence bounds for the system. To analyze the identification performance, Figure 5.6 presents the application ellipsoid for the parameter θ, together with the resulting identification ellipsoids and 50 identified parameters obtained with the optimal reference with probabilistic bounds, the optimal reference without probabilistic bounds, and for the random binary reference. From this figure we conclude that the 98% confidence level set for the identified parameters lies completely inside the application ellipsoid for all the external excitations. As expected, the confidence level set for the random binary reference is smaller than the ones obtained with the proposed technique, since the variance of this signal is greater than the one obtained with the optimal references. Hence, the random binary reference excites the system more than required, which makes the cost function in optimization problem (5.8) greater than the cost obtained with the proposed method. Indeed, the cost functions are J opt = 541.6 for the optimal experiment with probabilistic bounds, and J binary = 695.8 for a random binary reference, which is in line with the size of the uncertainty ellipsoids in Figure 5.6. On the other hand, we see that the confidence ellipsoids for the estimated parameters are almost the same when a external excitation is designed by including or excluding the probabilistic bounds on the input and output. 74 CHAPTER 5. CLOSED-LOOP INPUT DESIGN 0.85 θ3 0.8 0.75 0.7 0.65 0 0.05 0.1 θ1 0.15 0.2 0.25 Figure 5.7: Application ellipsoid (black, continuous line), with the respective identification ellipsoids. Red, dashed line: obtained identification ellipsoid from the optimal reference signal; Blue, dotted line: identification ellipsoid for a prbs. Example 2 This example is a simple model of two interconnected tanks. An upper tank is connected to a pump with input ut . There is hole in the tank with free flow to a lower tank, which also has a hole with free flow out of the tank. The lower tank level is the output, yt . An output error model of the two tank is θ θ4 4.5 x(t + 1) = 3 x(t) + u(t), 1 0 0 (5.36) y(t) = θ1 θ2 x(t) + e(t). The true parameter values are θo = [0.12 0.059 0.74 −0.14] and the noise variance is λe = 1. In this example we assume that θ2 and θ4 are known and we only want to identify θ1 and θ3 . The system is controlled in closed-loop using the following state feedback controller u(t) = r(t) − Ky x̂(t), (5.37) where Ky = [0.0756 − 0.0222]. The states are estimated using a simple observer. We aim to estimate the two unknown parameters θ = [θ1 θ3 ] in (5.36). Thus, we 5.4. NUMERICAL EXAMPLE 75 ut 3 −3 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 t 60 70 80 90 100 ut 3 −3 Figure 5.8: Part of the input signal {u(t)}500 t=1 . Top: Input signal for the optimal reference. Bottom: Input signal for a pseudo random binary reference. design an external excitation {r(t)}500 t=1 using the proposed approach in this chapter. We assume that r(t) is a realization from a stationary process with memory nm = 2 and r(t) ∈ C, where C := {−2, −1, 0, 1, 2}. The performance degradation is measured using the same application cost function as in (5.35), where y(t, θ) is the closed-loop output when θ is employed to design the observer and state feedback controller. We use the following parameters in Problem 5.2: Q = 1, R = 0.5, εy = εx = 0.1, ymax = 2.5, umax = 3, γ = 102 , and α = 0.98. The obtained application and identification ellipsoids are shown in Figure 5.7. It can be seen that the application ellipsoid embodies the identification ellipsoids for both optimal obtained external excitation and a pseudo random binary signal (prbs), with the same alphabet as r(t), as external excitation, which means both signals satisfied the experiment design constraint. However, the identification ellipsoid is oriented to the same direction as the application ellipsoid and thus the optimal external excitation excites the plant in the direction which is more important from applications point of view. Moreover, the obtained ellipsoid from random external excitation is smaller, and thus it excites the system more than required. The obtained input and output signals for both optimal external excitation and prbs are shown in Figures 5.8 and 5.9. The results of the probability measures for these realizations are summarized in Table 5.2. It is observed that for the optimal external excitation the input and output of the system will be in the acceptable range with probability of 96% and 91%, respectively, while these values are much 76 CHAPTER 5. CLOSED-LOOP INPUT DESIGN yt 2.5 −2.5 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 t 60 70 80 90 100 yt 2.5 −2.5 Figure 5.9: Part of the output {y(t)}500 t=1 . Top: Output signal for the optimal reference. Bottom: Output signal for a pseudo random binary reference. prbs r opt (t) P{|u(t)| ≤ umax } 81% 96% P{|y(t) − yd | ≤ ymax } 81% 91% Table 5.2: Probability measures of the input and output signals for the generated realization of the obtained optimal pdf and the prbs signal. smaller when we use a prbs signal with the same alphabet as r(t), without any optimization. 5.5 Conclusion In this chapter we extend application oriented experiment design to closed-loop system identification. A method is proposed to design input sequences for closedloop experiments with a general known control law. The method considers the input sequence as a realization of a stationary process minimizing the experimental cost, and subject to performance constraints. Using elements from graph-theory, the elements in the set of stationary processes are described as a convex combination of the measures associated with the prime cycles of the corresponding Brujin graph. Therefore, both experimental cost and constraints can be expressed as a convex combination of the values associated with the extreme measures in the set, which are 5.5. CONCLUSION 77 computed using numerical methods. An interesting feature of this approach is that probabilistic constraints become convex in the decision variables. The numerical example shows that this approach is an attractive method for the design of input sequences to identify models in a closed-loop setting. Chapter 6 Application Oriented Experiment Design for MPC In Chapter 5, we extend the application oriented experiment design to closedloop identification where the controller is known. In this chapter we focus on the case where the controller is MPC and there is no explicit control law. The proposed approach in Chapter 5 is not applicable to MPC in its current formulation since in the MPC the control law is not known in advance. However, due to the importance of MPC, closed-loop system identification for MPC has been studied in the literature. The authors in [18] has modified the MPC formulation to include a new constraint which forces the input to be persistently exciting over the control horizon. The method is called MPC with simultaneous model identification (MPCI). MPCI restricts the input signal to a periodic signal. The resulting formulation for (MPCI) is however nonconvex and numerical methods have been used to find local solutions for the problem. In [45] the authors introduced persistently exciting MPC (PE-MPC), which uses the same formulation as MPCI. The main difference is that PE-MPC ensures the persistently excitation constraints over a backward window of P samples where P is a tuning variable. There are a few other works that focuses on amending the persistently excitation constraint into the MPC formulation, see for example [49]. The main focus in the above mentioned algorithms is to generate inputs that are persistently exciting, which can ensure consistent parameter estimation. However, in application oriented experiment design the input signal is generated such that the estimated model is suitable for a specific application, which is not considered in generating persistently exciting input. The main advantage is that the intended application of the model will be considered and thus we avoid any unnecessary excitation of the system. Application oriented experiment design for MPC is studied in [33]. The proposed algorithm includes the experiment design constraint (2.29) in MPC formulation to get an extra excitation, which fulfils the identification requirements for the 79 80 CHAPTER 6. APPLICATION ORIENTED EXPERIMENT DESIGN FOR MPC intended use of the model on the input signal. The main advantage is that it is built on the existing MPC controllers. However, the resulting problem is nonconvex due to the experiment design constraint. Thus, a convex relaxation technique is also presented in [33]. Another drawback of the proposed approach is the risk for infeasible solutions, which might happen due to the the conflicting nature of the newly added experiment design constraint and control objectives. In this chapter a new formulation of the idea in [33] is considered to improve the algorithm and avoid infeasible solutions. The idea is to remove the possibility of infeasible MPC and at the same time get the highest possible excitation. We first explain “MPC with integrated experiment design for OE systems”, presented in [33]. We then propose a new formulation of this problem. 6.1 MPC with integrated experiment design for OE models In the application oriented experiment design framework, the authors in [33] consider the control cost (MPC cost function) as the cost of experiment. Their main focus is on OE models. They append the experiment design constraint to the MPC formulation to get extra excitations and solve the following optimization problem at each time instant t minimize J(t) = u {u(t+i)}N i=0 subject to Ny X kŷ(t + i|t) − r(t + i)k2Q + i=0 Nu X k∆u(t + i)k2R , i=0 x̂(t + i + 1|t) = A(θ)x̂(t + i|t) + B(θ)u(t + i), ŷ(t + i|t) = C(θ)x̂(t + i|t), i = 0, . . . , Nu ymin ≤ ŷ(t + i, |t) ≤ ymin , i = 0, . . . , Nu , umin ≤ u(t + i) ≤ umax , i = 0, . . . , Ny , x̂(t|t) = x∗ (t), ∆u(t) = u(t) − u∗ (t − 1), t+Ny IF (θ) κ(t) (6.1) γχ2α (nθ ) 00 Vapp (θ0 ). 2 Here x∗ (t) is the estimated system state obtained by an observer or direct measurement at time t, u∗ (t − 1) is the applied input signal at time t − 1, Ny and Nu are prediction and control horizon lengths, here for simplicity we assume that Nu = Ny . The matrices Q and R are adjustable weight matrices and κ(t) is a scaling factor that controls the excitation level at a given instant, see Section 2.4 at 20 for elaborated description of MPC parameters. The experiment design constraint is added to MPC formulation to ensure that over the MPC prediction horizon, the generated input signal can meet the applications specifications. The obtained optimization problem is nonconvex due to the experiment design constraint. The remedy is to relax the problem by introducing a lifting variable. 6.2. CLOSED LOOP SIGNAL GENERATION WITH BACK-OFF 81 We refer the reader to [33] for more details. The controller introduced in (6.1) is called MPC with extra excitation or MPC-X. The fundamental problem that arises by simply adding the experiment constraint to the problem is the risk for infeasible solutions. In other words, at each time instant t, satisfying the experiment design constraint might be in contradiction with the control objectives and system limitations. Thus, a trade-off between the experiment design constraint and the other requirements should be found. This is the main purpose of introducing the scaling factor κ(t). How to adjust the scaling factor optimally, is what we are going to answer in the rest of this chapter. The idea is to use κ(t) to back off from the identification requirements (experiment design constraint) when necessary and thus avoid the fact that the optimization problem becomes infeasible. At the same time we like to excite the system as much as possible. 6.2 Closed loop signal generation with back-off Consider again the experiment design constraint t+Ny IF (θ0 ) κ(t) γχ2α (nθ ) 00 Vapp (θ0 ). 2 (6.2) The scaling factor should be chosen such that the resulting Fisher information matrix after N samples fulfils the application requirements with probability of α. A large scaling factor is desirable (ideally it should be 1) from an identification perspective as it forces the controller to excite the system sufficiently. However, this may not be good from a control point of view, in particular when the process is close to signal constraints. In other words, an appropriate scaling factor should allow for adaptation of the signal waveform when the process is close to violating constraints. The back-off algorithm aims to find a trade-off between the identification and control performance by adapting the scaling factor. We start by noticing that t+Ny IF (θ0 ) κ(t) 00 ⇔ c(α)(Vapp (θ0 )) − 12 γχ2α (nθ ) 00 Vapp (θ0 ), 2 t+Ny IF 00 ⇔ c(α)λmin {(Vapp (θ0 )) 1 − 21 00 (θ0 )(Vapp (θ0 )) t+Ny IF (6.3) − 21 κ(t)I, 00 (θ0 )(Vapp (θ0 )) − 21 } κ(t), (6.4) (6.5) 00 00 where (Vapp (θ0 )) 2 is the Hermitian root square of Vapp (θ0 ), and c(α) is a constant term depending on α. We can maximize the upper bound on the scaling factor by solving the following 82 CHAPTER 6. APPLICATION ORIENTED EXPERIMENT DESIGN FOR MPC optimization: maximize u {u(t+i)}N i=0 00 λmin {(Vapp (θ0 )) − 21 t+Ny IF 00 (θ0 )(Vapp (θ0 )) − 21 } subject to x̂(t + i + 1|t) = A(θ)x̂(t + i|t) + B(θ)u(t + i), ŷ(t + i|t) = C(θ)x̂(t + i|t), i = 0, . . . , Nu ymin ≤ ŷ(t + i, |t) ≤ ymax , i = 0, . . . , Nu , umin ≤ u(t + i) ≤ umax , i = 0, . . . , Ny , x̂(t|t) = x∗ (t), ∆u(t) = u(t) − u∗ (t − 1), (6.6) The algorithm in (6.6) maximizes the scaling factor subject to the signal constraints. It thus adapts the scaling factor to the operating conditions. Control performance requirements The optimization problem (6.6) gives an upper bound on the scaling factor while considering the constraints on the input and output signals and the dynamics of the system. However, this formulation is still not able to meet the control performance requirements since the MPC cost is not considered. This problem can be addressed by adding the following constraint to the problem ∗ JM P C (t) ≤ JM P C (t) + ∆J, (6.7) ∗ JM P C (t), is the value of MPC cost function for the optimal solution to the where ordinary MPC with the same settings and without extra excitation constraints at time t, and ∆J is the maximum allowed increment in the MPC cost function. JM P C (t) is the MPC cost function which in our case is defined as follows JM P C (t) = Ny X kŷ(t + i|t) − r(t + i)k2Q + Nu X k∆u(t + i)k2R . (6.8) i=0 i=0 Proposed back-off algorithm The above considerations result in the following back-off algorithm: maximize u {u(t+i)}N i=0 00 λmin {(Vapp (θ0 )) − 21 t+Ny IF 00 (θ0 )(Vapp (θ0 )) − 21 } subject to x̂(t + i + 1|t) = A(θ)x̂(t + i|t) + B(θ)u(t + i), ŷ(t + i|t) = C(θ)x̂(t + i|t), i = 0, . . . , Nu ymin ≤ ŷ(t + i, |t) ≤ ymax , i = 0, . . . , Nu , umin ≤ u(t + i) ≤ umax , i = 0, . . . , Ny , x̂(t|t) = x∗ (t), ∆u(t) = u(t) − u∗ (t − 1), ∗ JM P C (t) ≤ JM P C (t) + ∆J. (6.9) 6.3. NUMERICAL EXAMPLE 83 This problem is not convex since the MPC cost is a quadratic function of the input signal. However, using the same approach as in [33], we can find a convex approximation of this optimization problem. The algorithm searches for the most exciting input signal which can satisfy the input and output constraints while at the same time guaranteeing that the performance of the controller, that is, the MPC cost function, does not deviate more than a pre-specified value, ∆J, from the standard MPC set-up. When the process is close to violating any of the constraints, the algorithm backs off by using a smaller bounds on the scaling factor, which in turn results in less exciting input signal, that is, less information is obtained. Note that the solution to the optimization problem (6.9) is an upper bound on the value of the scaling factor. It is worth mentioning that this algorithm can be used to find the minimum required time in order to satisfy the experiment design constraint, while at the same time we stay in a pre-specified interval around the optimal control performance. In this case, the minimum time will be the first time that the scaling factor can take the value of 1, which means the experiment design constraint has been satisfied completely. Thus, the algorithm is also known as minimum time MPC-X. 6.3 Numerical example In this section we evaluate the proposed algorithm on a simulation example from [33]. Example 1: avoiding infeasible solutions In this example we compare the proposed algorithm with MPC-X in terms of the ability to avoid infeasible solutions. Simulation settings Consider the following output error model of a two tank system: θ θ4 4.5 x(t + 1) = 3 x(t) + u(t), 1 0 0 y(t) = θ1 θ2 x(t) + e(t). (6.10) The true system parameters are given by [0.12 0.059 0.74 − 0.14]T . We aim to do reference tracking of the level in the lower tank, i.e. y(t), using MPC with the following settings: Ny = Nu = 5, Q = I, R = 0.001I. The initial model is obtained using the true parameter values. Let the application cost defined as Vapp (θ) = N 1 X 2 ky(t, θ0 ) − y(t, θ)k2 . N t=1 (6.11) 84 CHAPTER 6. APPLICATION ORIENTED EXPERIMENT DESIGN FOR MPC κ(t) 1 0.5 0 0 50 Time 100 150 Figure 6.1: The sample mean value of the scaling factor, κ(t), for a Monte Carlo with 20 trials in Section 6.3. The back-off in the scaling factor value is occurred when the set point value is shifted close to the constraint (t = 75). We set the length of the identification experiment to N = 150 samples and we assume that the desired accuracy is γ = 200 and the probability α is set to be 99%. We consider the case that the system is running in its steady state and the identification is performed around the equilibrium point while the maximum allowed values for input and output deviations are: −umin = umax = 2 and −ymin = ymax = 1, respectively. It is also assumed that the MPC cost function can be perturbed ∗ from its optimal value no more than 10%, i.e., ∆J(t) = 0.10JM P C (t). We perform a Monte Carlo trial with 20 simulations for both MPC-X suggested in [33], and the proposed back-off algorithm (minimum time MPC-X). We investigate the back-off effect by forcing the process to work close to the constraints. This is done by shifting the value of the set point near the output constraint at the time t = 75. Results The Monte Carlo trial results showed that the proposed back-off algorithm can deal with the infeasibility problem which stems from shifting the value of the set point. While the algorithm in [33] fails in tracking the reference signal in 19 simulations (out of 20 simulations), the proposed algorithm follows the set-point in all cases without giving infeasible solutions. It can be seen in the Figure 6.1 that when the reference signal is changed to a value closed to the constraint, the back-off algorithm decreases the scaling factor temporarily to avoid violating the constraint. The controller then adjusts itself to the new reference signal. After a few time steps the scaling factor again has been increased to generate input signals that can satisfy the experiment design constraint. At the end of the simulation time the scaling factor is 1, which means that the experiment design constraint has been satisfied completely. The input and output signals for MPC-X and MPC-X with back-off are shown in Figure 6.3 and 6.3. NUMERICAL EXAMPLE 85 u(t) 2 0 −2 0 50 0 50 Time 100 150 100 150 y(t) 1 0 −1 Time Figure 6.2: Input and output signals for one of the Monte Carlo simulations ofSection 6.3 when back-off algorithm is used. The output is tracking the reference signal without violating the constraints. Figure 6.2, respectively. We observe that after changing the reference signal close to the output bounds the MPC-X algorithm is not able to track the reference. Example 2: Finding minimum time In this example we use the proposed algorithm to find the minimum required time to satisfy the experiment design constraint. We then compare the results with MPC-X. Simulation settings In this example we again consider the two interconnected tank system (6.10). The goal is to control the level in the lower tank using MPC. The MPC parameters are set to be: Ny = Nu = 5, Q = 10I, R = I. We assume that the identification started when the plant is operating in steady state. The application cost is the same as the previous example with the same accuracy and probability, and the input is constrained to be between −1 and 1. No constraints are imposed on the output. 86 CHAPTER 6. APPLICATION ORIENTED EXPERIMENT DESIGN FOR MPC 2 u(t) 1 0 −1 −2 0 50 0 50 Time 100 150 100 150 1 y(t) 0.5 0 −0.5 −1 Time Figure 6.3: Input and output signal for one of the Monte Carlo simulations when MPC-X algorithm is used. The output is not able to track the set-point near the constraints. The considered scenario is to run the experiment until the experiment design constraint is satisfied. Results The results are summarized in Table 6.1. It is seen that MPC-X satisfies the experiment design constraint after 200 samples while the required number of samples for minimum time MPC-X is 95. The input variance for the minimum time MPC-X is 0.163 which is higher than the one obtained by MPC-X. However, the output variance for the minimum time MPC-X is higher. This is despite the fact that the PN total MPC cost, J = t=1 JMPC (t) where JMPC (t) is defined in (6.8), is lower for minimum time MPC-X. Therefore, minimum time MPC-X satisfies the experiment design constraint in a shorter time and with less control cost. 6.4 Conclusion In this chapter we studied MPC with integrated experiment design, MPC-X. We considered the OE model structure for the plant, in particular. A back-off al- 6.4. CONCLUSION Algorithm MPC-X minimum time MPC-X 87 var u 0.0793 0.1614 var y 0.0216 0.0315 N 200 95 J 130 85 Table 6.1: Comparison of MPC-X and minimum time MPC-X. The cost J is the total MPC cost over the whole experiment design. gorithm is proposed based on the idea in [33] to trade off between the control performance and identification objectives. The algorithm backs off from the identification requirements when it is close to the control constraints bounds, or the control performance deterioration is more than a specified user defined value. The main advantage is that we can avoid infeasible solutions for MPC-X. The method is also called minimum time MPC-X since one can find the minimum required time to satisfy the experiment design constraint. Chapter 7 Conclusion This thesis studied the problem of application oriented experiment design for system identification. The problem is usually formulated as a constrained optimization problem. The constraint guarantees that the set of the identified models should be a subset of the set that includes all acceptable models from applications point of view. While the former set is asymptotically an ellipsoid when PEM is employed during identification, the latter can be of any shape. Thus, the constraint is usually non-convex and requires a convex approximation of the application set. The thesis starts with studying this convex approximation problem. In Chapter 3 a new approach is proposed to find such an approximation, in particular for MPC. The main merit of the proposed method is that it finds analytic solution instead of numerical approximation and thus the computational load and time to calculate the approximation is comparably less than other existing methods. Once a convex approximation of the constraint is obtained the problem of application oriented experiment design is investigated under input and output bounds in Chapter 4. To handle these bounds which are expressed in time-domain, easily the experiment design problem is also formulated in time-domain. However, the obtained optimization problem is not convex. This issue is addressed employing alternating optimization techniques. This idea is evaluated by simulation examples and the results prove the capability of the method in handling the input and output constraints. In the rest of the thesis the problem of application oriented experiment design for closed-loop system identification is studied. The main burden here is the correlation between the output noise and the input signal. Two main problems are considered in Chapters 5 and 6. In the first problem, the plant is assumed to be controlled by a general (either linear or non-linear) but known controller. Here the problem is formulated as an optimization problem where the goal is to minimize the tracking error with minimum input energy by designing an additive external stationary input. The requirements on the identified parameters together with the probabilistic bound on the input and the resulting output are considered as constraints of the optimization 89 90 CHAPTER 7. CONCLUSION problem. The resulting problem is not convex. This issue is addressed by the help of graph theory, where the main idea is to characterize probability distribution function associated with the external excitation as the convex combination of the measures describing the set of stationary processes. The obtained problem is convex and easy to handle. In the second problem the loop is closed with MPC. Here an existing solution for the problem is considered. The main idea is to add the set constraint to the MPC constraint and then find a convex approximation of the problem by the help of lifting variable. The main difficulty that appears is however, the possibility of obtaining infeasible solutions due to the newly added constraint. To address this problem a back-off algorithm is proposed which is trying to find a trade off between the set constraint and the control performance. The algorithm avoids infeasible solutions by backing off from the identification requirements. This is done by optimizing a scaling factor for the set constraint. The method is evaluated by a simulation example and the results show that when the process is working closed to the control constraints, the algorithm backs off from the set constraint to avoid infeasible solutions. As a conclusion, in this thesis we studied the problem application oriented experiment design and our main contribution is considering constraints in time domain for the application oriented experiment design both in open-loop and closedloop frameworks. 7.1 Future works Several interesting problems are still open in application oriented experiment design that need to be addressed. Generally, how should one choose a good measure of model quality in different applications? It is of great importance to find an application cost which suits both real industrial demands and the theoretical framework. Currently, we are using a simulation-based application cost which is appropriate for reference tracking problems, however, the application cost could be a measure of economic cost or production quality. One can study the effects of choosing different application cost functions Application set approximation The proposed method in Chapter 3 computes the second order derivative of the application function, analytically. The method relies on the second order Taylor expansion of the application function. In many cases the second order approximation is not good enough to cover the application set. The idea in Chapter 3 can be extended towards constructions of higher order Taylor expansions of the application cost function. 7.1. FUTURE WORKS 91 Application oriented experiment design for closed-loop identification The proposed method for closed-loop input design in Chapter 5 considers a linear model structure for the plant, under a known nonlinear feedback. However, the method is not applicable to the cases with unknown control law such as MPC. Thus, an extension of the method to include implicit feedback laws is a future research direction in the topic. Minimum time MPC-X The main idea in minimum time MPC-X is to trade of between the control performance and identification requirements. Perhaps one can compare the control cost we are paying (e.g. in terms of performance degradation) with the obtained improvement in the identification due to the extra excitation. Bibliography [1] Autoprofit. URL http://www.fp7-autoprofit.eu/home/. [2] Debate Europe-building on the experience of plan D for democracy, dialogue and debate. European Economic and Social Committee and the Committee of the Regions, COM 158/4, 2008. [3] Märta Barenthin Syberg. Complexity Issues, Validation and Input Design for Control in System Identification. Phd thesis, Royal Institute of Technology (KTH), Nov 2008. [4] Xavier Bombois, Gérard Scorletti, Michel Gevers, Paul Van den Hof, and Ronald Hildebrand. Least costly identification experiment for control. Automatica, 42(10):1651–1662, October 2006. [5] J. Frédéric Bonnans and Alexander Shapiro. Optimization problems with perturbations: A guided tour. SIAM Rev., 40(2):228–264, June 1998. ISSN 0036-1445. [6] Stephen Boyd, Laurent El Ghaoui, Eric Feron, and Venkataraman Balakrishnan. Linear matrix inequalities in system and control theory. Studies in Applied Mathematics SIAM, 15, 1994. [7] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2012. [8] Chiara Brighenti, Bo Wahlberg, and Cristian R. Rojas. Input design using markov chains for system identification. In Proceedings of the 48th IEEE Conference on Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009, pages 1557–1562, 2009. [9] Giuseppe Carlo Calafiore and Marco Claudio Campi. The Scenario Approach to Robust Control Design. IEEE Transactions on Automatic Control, 51(5): 742–753, may 2006. ISSN 0018-9286. [10] Marco Claudio Campi and Simone Garatti. The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs. SIAM Journal on Optimization, 19(3):1211–1230, 2008. 93 94 BIBLIOGRAPHY [11] Xi-Ren Cao. Stochastic learning and optimization : A sensitivity-based approach. Annual Reviews in Control, 33(1):11–24, april 2009. ISSN 13675788. [12] Paul Van den Hof. Closed-loop issues in system identification. Annual Reviews in Control, 22(0):173 – 186, 1998. ISSN 1367-5788. [13] John D´Errico. DERIVESTsuite, 2007. mathworks. [14] Saurabh Deshpande, Bala Srinivasan, and Dominique Bonvin. How Important is the Detection of Changes in Active Constraints in Real-Time Optimization. In 18th IFAC World Congress, 2011. [15] Valerii V Fedorov. Theory of optimal experiments. Academic Press, 1972. [16] Marco Forgione, Xavier Bombois, Paul M.J. Van den Hof, and Håkan . Hjalmarsson. Experiment design for parameter estimation in nonlinear systems based on multilevel excitation. In European control conference (submitted to), 2014. [17] Urban Forssell and Lennart Ljung. Closed-loop identification revisited. Automatica, 35:1215–1241, 1999. [18] Hasmet Genceli and Michael Nikolaou. New approach to constrained predictive control with simultaneous model identification. AIChE Journal, 42(10):2857– 2868, Oct 1996. [19] Laszlo Gerencser, Håkan Hjalmarsson, and Jonas Mårtensson. Identification of arx systems with non-stationary inputs - asymptotic analysis with application to adaptive input design. Automatica, 45(3):623–633, March 2009. [20] Michel Gevers and Lennart Ljung. Optimal experiment design with respect to the intended model application. Automatica, 22:543–554, 1986. [21] G. Goodwin and R.L. Payne. Dynamic System Identification: Experiment Design and Data Analysis. Academic Press, New York, 1977. [22] Uday G. Gujar and R. Kavanagh. Generation of random signals with specified probability density functions and power density spectra. Automatic Control, IEEE Transactions on, 13(6):716–719, dec 1968. [23] Didier Henrion and Jerome Malick. Projection methods in conic optimization. Handbook on Semidefinite, Conic and Polynomial Optimization, 166:565–600, 2012. [24] Roland Hildebrand and Gabriel Solari. Closed-loop optimal input design: The partial correlation approach. In 15th IFAC Symposium on System Identification, Saint-Malo, France, July 2009. BIBLIOGRAPHY 95 [25] Ronald Hildebrand and Michel Gevers. Identification for control: optimal input design with respect to a worst-case ν-gap cost function. SIAM Journal on Control and Optimization, 41:1586–1608, March 2003. [26] Håkan Hjalmarsson. From experiment design to closed-loop control. Automatica, 41(3):393–438, March 2005. [27] Håkan Hjalmarsson. System Identification of Complex and structured systems. European Journal of Control, 15(3):275–310, 2009. [28] Håkan Hjalmarsson and Freja Egebrand. Input design using cylindrical algebraic decomposition. In IEEE Conference on Decision and Control and European Control Conference, pages 811–817, Orlando, FL, USA, december 2011. ISBN 978-1-61284-801-3. [29] Håkan Hjalmarsson and Henrik Jansson. Closed loop experiment design for linear time invariant dynamical systems via LMIs. automatica, 44:623–636, 2008. [30] Yu-Chi HO. Parametric Sensitivity of a Statistical Experiment. IEEE Transactions on Automatic Control, AC-24(6):982–983, 1979. [31] Henrik Jansson. Experiment design with applications in identification for control. Phd thesis, Royal Institute of Technology (KTH), Dec 2004. [32] Henrik Jansson and Håkan Hjalmarsson. Input design via lmis admitting frequencywise model specifications in confidence regions. automatica, 50:1534– 1549, 2005. [33] Christian A Larsson, Mariette Annergren, Hå kan Hjalmarsson, Cristian R Rojas, Xavier Bombois, Ali Mesbah, and Per Erik Modén. Model predicitive control with integrated experiment design for output error systems. In European Control Conference, pages 3790–3795, Zurich, 2013. ISBN 9783952417348. [34] Christian. A. Larsson, Mariette Annergren, and Hakan Hjalmarsson. On Optimal Input Design in System Identification for Model Predictive Control. In IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pages 805–810, Orlando, FL, USA, 2011. [35] Christian. A Larsson, Per Hägg, and Håkan Hjalmarsson. Generation of excitation signals with prescribed autocorrelation for input and output constrained systems. In European Control Conference, Zr"uich, Switzerland, July 2013. IEEE. [36] Christian A Larsson, Cristian Rojas, and Håkan Hjalmarsson. MPC oriented experiment design. In 18th IFAC World Congress, number 1986, pages 9966– 9971, Milano (Italy) August, 2011. 96 BIBLIOGRAPHY [37] Christian.A Larsson and Mariette Annergren. Moose: Model based optimal input signal design toolbox for matlab. Technical report, Royal Institute of Technology (KTH), 2011. [38] Alan J. Laub. Matrix Analysis for Scientists and Engineers. Society of Industrial and Applied Mathematics (SIAM), USA, 2005. [39] Kristian Lindqvist and Håkan Hjalmarsson. Identification for control: Adaptive input design using convex optimization. In In Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, US, 2001. [40] Bede Liu and David C. Munson Jr. Generation of a random sequence having a jointly specified marginal distribution and autocovariance. Acoustics, Speech and Signal Processing, IEEE Transactions on, 30(6):973–983, dec 1982. [41] Lennart Ljung. System Identification - Theory for the user. Prentice Hall, New Jersey, 2nd edition, 1999. [42] J. Ward MacArthur and Charles Zhan. A practical global multi-stage method for fully automated closed-loop identification of industrial processes. Journal of Process Control, 17(10):770 – 786, 2007. ISSN 0959-1524. [43] Maciejowski, Jan M. Predictive Control with Constraints. Edinburgh Gate, Harlow, Essex, England: Prentice Hall, 2002. [44] Ian R. Manchester. Input Design for System Identification via Convex Relaxation. In 49th IEEE Conference on Decision and Control (CDC 2010), USA, 2010. [45] Giancarlo Marafiato. Enchanced model predictive control: dual control approach and state estimation issues. Phd thesis, Norwegian University of Science and Technology (NTNU), Department of Engineerign Cybernetics, 2010. [46] Manfred Morari and Jay H. Lee. Model predictive control: Past, present and future. Computers & Chemical Engineering, 23(4):667–682, 1999. [47] Maria Prandini, Simone Garatti, and John Lygeros. A Randomized Approach to Stochastic Model Predictive Control. In IEEE Conference on Decision and Control, Maui, Hawaii, USA, December 2012. [48] S Joe Qin and Thomas A Badgwell. A survey of industrial model predictive control technology. Control engineering practice, 11(7):733–764, 2003. [49] Jan Rathouský and Vladimir Havlena. Mpc-based approximate dual controller by information matrix maximization. International Journal of Adaptive Control and Signal Processing, 27(11):974–999, 2013. ISSN 1099-1115. [50] R.Tyrrell Rockafellar. Convex Analysis. Princeton University Press, Englewood Cliffs, New Jersey, 1970. BIBLIOGRAPHY 97 [51] Cristian R. Rojas, Dimitrios Katselis, Håkan Hjalmarsson, Roland Hildebrand, and Mats Bengtsson. Chance constrained input design. In IEEE Conference on Decision and Control and European Control Conference, number 5, pages 2957–2962. IEEE, december 2011. ISBN 978-1-61284-801-3. [52] Cristian. R Rojas, James Welsh, Graham Goodwin, and Arie Feuer. Robust optimal experiment design for system identification. Automatica, 43:993–1008, 2007. [53] M. Schroeder. Synthesis of low-peak-factor signals and binary sequences with low autocorrelation. Information Theory, IEEE transactions on, 16:85–89, Jan 1970. [54] Stale A Skogestad. Dynamics and control of distillation columns: A tutorial introduction. Chemical Engineering Research and Design, 75(6):539 – 562, 1997. ISSN 0263-8762. [55] Torsten Söderström and Peter Stoica. System Identification. Prentice Hall, Englewood Cliffs, New Jersey, 1989. [56] James C. Spall. Adaptive stochastic approximation by the simultaneous perturbation method. Automatic Control, IEEE Transactions on, 45(10):1839–1853, Oct 2000. ISSN 0018-9286. [57] James C. Spall. Improved methods for Monte Carlo estimation of the Fisher Information Matrix, 2008. [58] Peter Stoica and Torsten Söderström. A useful input parameterization for optimal experiment design. IEEE Transactions on Automatic Control, 27(4): 986–989, August 1982. ISSN 0018-9286. [59] Petre Stoica, Jian Li, Xumin Zhu, and Bin Guo. Waveform Synthesis for Diversity-Based Transmit Beampattern Design. IEEE Transactions on Signal Processing, 56(6):2593–2598, 2008. [60] Joel A. Tropp, Inderjit S Dhillon, Robert W Heath Jr, and Tomas Strohmer. designing structured tight frames via an alternating projection method. IEEE Transactions on Information Theory, 51:188 – 209, Jan 2005. [61] Patricio E. Valenzuela, Johan Dahlin, Cristian R. Rojas, and Tomas Schön. A graph/particle-based method for experiment design in nonlinear systems. Accepted for publication in the 19th IFAC World Congress, Cape Town, South Africa. arXiv:1403.4044, March 2014. [62] Patricio E Valenzuela, Cristian R. Rojas, and Håkan Hjalmarsson. for nonlinear dynamic systems: a graph theory approach. In 52th IEEE Conference on Decision and Control (CDC’13), Florence, Italy, arXiv:1310.4706, Dec 2013. 98 BIBLIOGRAPHY [63] Patricio Esteban Valenzuela. Optimal input design for nonlinear dynamical systems: a graph-theory approach. November 2014. [64] Paul Van den Hof and Raymond A. de Callafon. Multivariable closed-loop identification: from indirect identification to dual-youla parametrization. In Proceedings of the 35th IEEE Conference on Decision and Control, volume 2, pages 1397–1402 vol.2, Dec 1996. [65] Paul M.J. Van den Hof and Ruud J. P. Schrama. An indirect method for transfer function estimation from closed loop data. Automatica, 29:1523 – 1527, 1993. [66] Bo Wahlberg, Hakan Hjalmarson, and Mariette Annergren. On Optimal Input Design in System Identification for Control. In 49th IEEE Conference on Decision and Control, Atlanta, GA, USA, 2010. [67] Federico Zenith, J. Tjonnas, Ingrid Schjolberg, Bjarne FOSS, and Vidar GUNNERUD. Well testing, May 2013. CA Patent App. CA 2,856,128. [68] Yucai Zhu. System identification for process control: recent experience and outlook. Int. Journal of Modelling, Identification and Control, 6(2):89–103, 2009. [69] Yucai Zhu. System identification for process control: recent experience and outlook. International Journal of Modelling, Identification and Control, 6, 2009.

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement