Read an Excerpt
Optimal Control
Linear Quadratic Methods
By Brian D. O. Anderson, John B. Moore Dover Publications, Inc.
Copyright © 1990 Brian D. O. Anderson and John B. Moore
All rights reserved.
ISBN: 978-0-486-15157-1
CHAPTER 1
Introduction
1.1 LINEAR OPTIMAL CONTROL
The methods and techniques of what is now known as "classical control" will be familiar to most readers. In the main, the systems or plants that can be considered by using classical control ideas are linear and time invariant, and have a single input and a single output. The primary aim of the designer using classical control design methods is to stabilize a plant, whereas secondary aims may involve obtaining a certain transient response, bandwidth, disturbance rejection, steady state error, and robustness to plant variations or uncertainties. The designer's methods are a combination of analytical ones (e.g., Laplace transform, Routh test), graphical ones (e.g., Nyquist plots, Nichols charts), and a good deal of empirically based knowledge (e.g., a certain class of compensator works satisfactorily for a certain class of plant). For higher-order systems, multiple-input systems, or systems that do not possess the properties usually assumed in the classical control approach, the designer's ingenuity is generally the limiting factor in achieving a satisfactory design.
Two of the main aims of modern, as opposed to classical, control are to de-empiricize control system design and to present solutions to a much wider class of control problems than classical control can tackle. One of the major ways modern control sets out to achieve these aims is by providing an array of analytical design procedures that facilitate the design task.
In the early stages of a design, the designer must use his familiarity with the engineering situation, and understanding of the underlying physics, to formulate a sensible mathematical problem. Then the analytical design procedures, often implemented these days with commercial software packages, yield a solution—which usually serves as a first cut in a trial and error iterative process.
Optimal control is one particular branch of modern control that sets out to provide analytical designs of a specially appealing type. The system that is the end result of an optimal design is not supposed merely to be stable, have a certain bandwidth, or satisfy any one of the desirable constraints associated with classical control, but it is supposed to be the best possible system of a particular type—hence, the word optimal. If it is both optimal and possesses a number of the properties that classical control suggests are desirable, so much the better.
Linear optimal control is a special sort of optimal control. The plant that is controlled is assumed linear, and the controller, the device that generates the optimal control, is constrained to be linear. Linear controllers are achieved by working with quadratic performance indices. These are quadratic in the control and regulation/tracking error variables. Such methods that achieve linear optimal control are termed LinearQuadratic (LQ) methods. Of course, one may well ask: why linear optimal control, as opposed simply to optimal control? A number of justifications may be advanced; for example, many engineering plants are linear prior to addition of a controller to them; a linear controller is simple to implement physically, and will frequently suffice.
Other advantages of optimal control, when it is specifically linear, follow.
1. Many optimal control problems do not have computable solutions, or they have solutions that may be obtained only with a great deal of computing effort. By contrast, nearly all linear optimal control problems have readily computable solutions.
2. Linear optimal control results may be applied to nonlinear systems operating on a small signal basis. More precisely, suppose an optimal control has been developed for some nonlinear system with the assumption that this system will start in a certain initial state. Suppose, however, that the system starts in a slightly different initial state, for which there exists some other optimal control. Then a first approximation to the difference between the two optimal controls may normally be derived, if desired, by solving a linear optimal control problem (with all its attendant computational advantages). This holds independently of the criterion for optimality for the nonlinear system. (We list two references and that outline this important result.)
3. The computational procedures required for linear optimal design may often be carried over to nonlinear optimal problems. For example, the nonlinear optimal design procedures based on the theory of the second variation and quasilinearization consist of computational algorithms replacing the nonlinear problem by a sequence of linear problems.
4. Linear optimal control designs where the plant states are measurable turn out to possess a number of properties, other than simply optimality of a quadratic index, which classical control suggests are attractive. Examples of such properties are good gain margin and phase margin, and good tolerance of nonlinearities. Such robustness properties can frequently be achieved even when state estimation is required. The robustness properties suggest that controller designs for nonlinear systems may sometimes be achieved by designing with the assumption that the system is linear (even though this may not be a good approximation), and by relying on the fact that an optimally designed linear system can tolerate nonlinearities—actually quite large ones—without impairment of all its desirable properties. Hence, linear optimal design methods are in some ways applicable to nonlinear systems.
5. Linear optimal control provides a framework for the unified treatment of the control problems studied via classical methods. At the same time, it vastly extends the class of systems for which control designs may be achieved.
Linear optimal control design for time-invariant systems is largely a matter of control law synthesis; see the flow chart of Figure 1.1-1 for the approach emphasized in this text. Recall that the designer's first task is to use his or her engineering understanding to formulate a mathematical problem. This is embodied in the top two blocks. If we disregard the iteration in this and later steps (the iterations are illustrated in the flowchart), there are three essential steps covered by the modern analytical procedures of this text. These are
full-state feedback design (where it is assumed that all states are measured and available for feedback)
state estimator design (where the concern is to estimate values of the states when they cannot all be measured directly, but certain measurements are available)
controller reduction (where the concern is to approximate a complicated state estimate feedback controller obtained from the above two steps by a simpler one—complication usually being measured by the state dimension)
The final major stage of design, involving the implementation of the controller, may involve the derivation of a discrete-time approximation to the controller.
In the second step (state estimator design), a variation is to estimate only the state feedback control signal, rather than the full state vector.
Linear quadratic methods that from the start build in controller constraints such as controller order are dealt with only briefly in this text. For full details see, for example.
1.2 ABOUT THIS BOOK IN PARTICULAR
This is not a book on optimal control, but a book on optimal control via linear quadratic methods. Accordingly, it reflects very little of the techniques or results of general optimal control. Rather, we study a basic problem of linear optimal control, the "regulator problem," and attempt to relate mathematically all other problems discussed to this one problem. If the reader masters the mathematics of the regulator problem, he should find most of the remainder of the mathematics easy going.
We aim to analyze the engineering properties of the solution to the problems presented. We thus note the various connections to classical control results and ideas, which, in view of their empirical origins, are often best for providing a framework for a modern control design and assessing a practical design.
1.3 PART AND CHAPTER OUTLINE
In this section, we briefly discuss the breakdown of the book into parts and chapters. There are three parts, listed below with brief comments.
Part I—Basic theory of the optimal regulator. These chapters serve to introduce the linear regulator problem and to set up the basic mathematical results associated with it. Chapter 1 is introductory. Chapter 2 sets up the problem by translating into mathematical terms the physical requirements on a regulator. It introduces the Principle of Optimality and the Hamilton-Jacobi equation for solving optimal control problems, and then obtains a solution for problems where performance over a finite (as opposed to infinite) time interval is of interest. The infinite-time interval problem is considered in Chapter 3, which includes stability properties of the optimal regulators, and shows how to achieve a regulator design with a prescribed degree of stability. Also considered is the formulation of an optimal linear regulator problem by linearization of a nonlinear system and computation of the second variation of an optimized index. Chapter 4 considers tracking problems by building on the regulator theory. In tracking, one generally wishes the plant output to follow a specific prescribed time function or a signal from a class, for example, a step function of unknown magnitude.
Part II—Properties of the optimal regulator. In Chapter 5, frequency domain formulas are derived to deduce sensitivity and robustness properties. In particular, the return difference relation is studied along with its interpretation as a spectral factorization. Robustness measures in terms of sensitivity and complementary sensitivity functions are introduced, and for the multivariable case, the role of singular values is explored. Gain and phase margins and tolerance of sector nonlinearities are optimal regulator properties studied. The inverse problem of optimal control is briefly mentioned. In Chapter 6, the relationship between quadratic index weight selection and closed-loop properties is studied, with emphasis on the asymptotic properties as the control cost weight approaches infinity or zero.
Part III—State estimation and linear-quadratic-gaussian design. When the states of a plant are not available, then the certainty equivalence principle suggests that state estimates be used instead of states in a state feedback design. Chapter 7 deals with state estimation, including the case when measurements are noisy, with, in the ideal case, additive gaussian noise. Design methods and properties are developed, to some extent exploiting the theory of Parts I and II. Chapter 8 deals with control law synthesis using full state feedback designs and state estimation. It is pointed out that when the plant is precisely known, then in the linear-quadratic-gaussian (LQG) case, certainty equivalence is the optimal approach. This is the separation theorem. Otherwise, there can be poor robustness properties, unless loop recovery and frequency shaping techniques are adopted, as studied in Chapters 8 and 9, respectively. State estimate feedback designs, particularly when frequency shaped, may result in controllers of unacceptably high order. Controller reduction methods are studied in Chapter 10. These attempt to maintain controller performance and robustness properties while reducing controller complexity. Finally, in Chapter 11, some practical aspects concerning implementation of controllers via digital computers are studied.
Appendices. Results in matrix theory, linear system theory, the Minimum Principle, stability theory and Riccati equations relevant to the material in the book are summarized in the appendices.
CHAPTER 2
The Standard Regulator Problem—1
2.1 A REVIEW OF THE REGULATOR PROBLEM
We shall be concerned almost exclusively with linear finite-dimensional systems, which frequently will also be time invariant. The systems may be represented by equations of the type
[??](t) = F(t)x(t) + G(t)u(t) (2.1-1)
y(t) = H'(t)x(t) (2.1-2)
Here, F(t), G(t), and H(t) are matrix functions of time, in general with continuous entries. If their dimensions are respectively n × n, n × m, n × p, the n vector x (t) denotes the system state at time t, the m vector u (t) the system input or system control at time t, and the p vector y (t) the system output at time t. The superscript prime denotes matrix transposition.
In classical control work, usually systems with only one input and output are considered. With these restrictions in (2.1-1) and (2.1-2), the vectors u(t) and y(t) become scalars, and the matrices G(t) and H(t) become vectors, and accordingly will often be denoted by lowercase letters to distinguish their specifically vector character. The systems considered are normally also time-invariant. In terms of (2.1-1) and (2.1-2), this means that the input u(t) and output y(t) for an initially zero state are related by a time-invariant impulse response. Furthermore, the most common state-space descriptions of time-invariant systems are those where F(t), g(t), and h(t) are constant with time. Note, though, that nonconstant F(t), g(t), and h(t) may still define a time-invariant impulse response—e.g., F(t) = 0, g(t) = et h(t) = e-t defines a time-invariant impulse response via the map
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
The classical description of a system is normally in terms of its transfer function matrix, which we denote by W (s), s being the Laplace transform variable. The well-known connection between W (s) and the matrices of (2.11) and (2.1-2), if these are constant, is
W(s) = H'(sI - F)-1G (2.1-3)
A common class of control problems involves a plant, for which a control is desired to achieve one of the following aims:
1. Qualitative statement of the regulator problem. Suppose that initially the plant output, or any of its derivatives, is nonzero. Provide a plant input to bring the output and its derivatives to zero. In other words, the problem is to apply a control to take the plant from a nonzero state to the zero state. This problem may typically occur where the plant is subjected to unwanted disturbances that perturb its output (e.g., a radar antenna control system with the antenna subject to wind gusts).
2. Qualitative statement of the tracking (or servomechanism) problem. Suppose that the plant output, or a derivative, is required to track some prescribed function. Provide a plant input that will cause this tracking (e.g., when a radar antenna is to track an aircraft, such a control is required).
In a subsequent chapter, we shall discuss the tracking problem. For the moment, we restrict our attention to the more fundamental regulator problem; thus no external input is applied.
When considering the regulator problem using classical control theory, we frequently seek a solution that uses feedback of the output and its derivatives to generate a control. A controller with a transfer function description is interposed between the plant output and plant input. The plant output is the controller input, and the controller output is the plant input. The feedback arrangement is shown in Fig. 2.1-1. Both the plant and controller have a single input and output, and are time-invariant. Each possesses a transfer function.
In the optimal control approach of this text, it is assumed in the first instance that the plant states are available for measurement. If this is not the case, it is generally possible to construct a physical device called a state estimator driven by both the plant input and output. This produces at its output estimates of the plant states, and these may be used in lieu of the states. This will be discussed in a later chapter. In addition to assuming availability of the states, it is usual in the first instance to seek controllers that are nondynamic, or memory less. In other words, the controller output or plant input u (t) is assumed to be an instantaneous function of the plant state x (t). The nature of this function may be permitted to vary with time, in which case we could write down a control law
u(t) = k(x(t),t) (2.1-4)
to indicate the dependence of u (t) on both x (t) and t.
(Continues...)
Excerpted from Optimal Control by Brian D. O. Anderson, John B. Moore. Copyright © 1990 Brian D. O. Anderson and John B. Moore. Excerpted by permission of Dover Publications, Inc..
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
