Introduction to Optimal Estimation

The topics of control engineering and signal processing continue to flourish and develop. In common with general scientific investigation, new ideas, concepts and interpretations emerge quite spontaneously and these are then discussed, used, discarded or subsumed into the prevailing subject paradigm. Sometimes these innovative concepts coalesce into a new sub-discipline within the broad subject tapestry ofcontrol and signal processing. This preliminary batde between old and new usually takes place at conferences, through the Internet and in the journals of the discipline. After a litde more maturity has been acquiredhas been acquired by the new concepts then archival publication as ascientificorengineering monograph mayoccur. Anewconceptin control and signal processing is known to have arrived when sufficient material has developed for the topic to be taught as a specialised tutorial workshop or as a course to undergraduates, graduates or industrial engineers. The Advanced Textbooks in Control and Signal Processing Series is designed as a vehicle for the systematic presentation ofcourse material for both popular and innovative topics in the discipline. It is hoped that prospective authors will welcome the opportunity to publish a structured presentation of either existing subject areas or some of the newer emerging control and signal processing technologies.

1 Introduction.- 1.1 Signal Estimation.- 1.2 State Estimation.- 1.3 Least Squares Estimation.- Problems.- 2 Random Signals and Systems with Random Inputs.- 2.1 Random Variables.- 2.2 Random Discrete-Time Signals.- 2.3 Discrete-Time Systems with Random Inputs.- Problems.- 3 Optimal Estimation.- 3.1 Formulating the Problem.- 3.2 Maximum Likelihood and Maximum a posteriori Estimation.- 3.3 Minimum Mean-Square Error Estimation.- 3.4 Linear MMSE Estimation.- 3.5 Comparison of Estimation Methods.- Problems.- 4 The Wiener Filter.- 4.1 Linear Time-Invariant MMSE Filters.- 4.2 The FIR Wiener Filter.- 4.3 The Noncausal Wiener Filter.- 4.4 Toward the Causal Wiener Filter.- 4.5 Derivation of the Causal Wiener Filter.- 4.6 Summary of Wiener Filters.- Problems.- 5 Recursive Estimation and the Kaiman Filter.- 5.1 Estimation with Growing Memory.- 5.2 Estimation of a Constant Signal.- 5.3 The Recursive Estimation Problem.- 5.4 The Signal/Measurement Model.- 5.5 Derivation of the Kaiman Filter.- 5.6 Summary of Kaiman Filter Equations.- 5.7 Kaiman Filter Properties.- 5.8 The Steady-state Kaiman Filter.- 5.9 The SSKF as an Unbiased Estimator.- 5.10 Summary.- Problems.- 6 Further Development of the Kaiman Filter.- 6.1 The Innovations.- 6.2 Derivation of the Kaiman Filter from the Innovations.- 6.3 Time-varying State Model and Nonstationary Noises.- 6.4 Modeling Errors.- 6.5 Multistep Kaiman Prediction.- 6.6 Kaiman Smoothing.- Problems.- 7 Kaiman Filter Applications.- 7.1 Target Tracking.- 7.2 Colored Process Noise.- 7.3 Correlated Noises.- 7.4 Colored Measurement Noise.- 7.5 Target Tracking with Polar Measurements.- 7.6 System Identification.- Problems.- 8 Nonlinear Estimation.- 8.1 The Extended Kalman Filter.- 8.2 An Alternate Measurement Update.- 8.3 Nonlinear System Identification UsingNeural Networks.- 8.4 Frequency Demodulation.- 8.5 Target Tracking Using the EKF.- 8.6 Multiple Target Tracking.- Problems.- A The State Representation.- A.1 Discrete-Time Case.- A.2 Construction of State Models.- A.3 Dynamical Properties.- A.4 Discretization of Noise Covariance Matrices.- B The z-transform.- B.1 Region of Convergence.- B.2 z-transform Pairs and Properties.- B.3 The Inverse z-transform.- C Stability of the Kaiman Filter.- C.1 Observability.- C.2 Controllability.- C.3 Types of Stability.- C.4 Positive-Definiteness of P(n).- C.5 An Upper Bound for P(n).- C.6 A Lower Bound for P(n).- C.7 A Useful Control Lemma.- C.8 A Kaiman Filter Stability Theorem.- C.9 Bounds for P(n).- D The Steady-State Kaiman Filter.- D.2 A Stabilizability Lemma.- D.3 Preservation of Ordering.- D.5 Existence and Stability.- E Modeling Errors.- E.1 Inaccurate Initial Conditions.- E.2 Nonlinearities and Neglected States.- References.