# best linear unbiased estimator proof

To show this property, we use the Gauss-Markov Theorem. The Gauss-Markov theorem states that if your linear regression model satisfies the first six classical assumptions, then ordinary least squares regression produces unbiased estimates that have the smallest variance of all possible linear estimators..  Rao, C. Radhakrishna (1967). is the Best Linear Unbiased Estimator (BLUE) if εsatisﬁes (1) and (2). Proof: E[b0Y] = b0Xβ, which equals a0β for all β if and only if a = X0b. Deﬁnition: A linear combination a0β is estimable if it has a linear unbiased estimate, i.e., E[b0Y] = a0β for some b for all β. Lemma 10.2.1: (i) a0β is estimable if and only if a ∈ R(X0). Two matrix-based proofs that the linear estimator Gy is the best linear unbiased estimator. Goldsman — ISyE 6739 12.2 Fitting the Regression Line Then, after a little more algebra, we can write βˆ1 = Sxy Sxx Fact: If the εi’s are iid N(0,σ2), it can be shown that βˆ0 and βˆ1 are the MLE’s for βˆ0 and βˆ1, respectively. Except for Linear Model case, the optimal MVU estimator might: 1. not even exist 2. be difficult or impossible to find ⇒ Resort to a sub-optimal estimate BLUE is one such sub-optimal estimate Idea for BLUE: 1. BLUE. If all Gauss-Markov assumptions are met than the OLS estimators alpha and beta are BLUE – best linear unbiased estimators: best: variance of the OLS estimator is minimal, smaller than the variance of any other estimator linear: if the relationship is not linear – OLS is not applicable. 11 The proof for this theorem goes way beyond the scope of this blog post. A property which is less strict than efficiency, is the so called best, linear unbiased estimator (BLUE) property, which also uses the variance of the estimators. \$\begingroup\$ It is the best filter in the sense of minimizing the MSE; However, it is not necessarily unbiased. In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE. Proof: An estimator is “best” in a class if it has smaller variance than others estimators in the same class. sometimes called best linear unbiased estimator Estimation 7–21. \$\endgroup\$ – Dovid Apr 23 '18 at 14:47 ... they go on to prove the best linear estimator property for the Kalman filter in Theorem 2.1, and the proof does not … If the estimator is both unbiased and has the least variance – it’s the best estimator. with minimum variance) A vector of estimators is BLUE if it is the minimum variance linear unbiased estimator. If the estimator has the least variance but is biased – it’s again not the best! Puntanen, Simo; Styan, George P. H. and Werner, Hans Joachim (2000). Properties of Least Squares Estimators Each ^ iis an unbiased estimator of i: E[ ^ i] = i; V( ^ i) = c ii˙2, where c ii is the element in the ith row and ith column of (X0X) 1; Cov( ^ i; ^ i) = c ij˙2; The estimator S2 = SSE n (k+ 1) = Y0Y ^0X0Y n (k+ 1) is an unbiased estimator of ˙2. I got all the way up to 11.3.18 and then the next part stuck me. Restrict estimate to be unbiased 3. Best Linear Unbiased Estimators. We now consider a somewhat specialized problem, but one that fits the general theme of this section. Now, talking about OLS, OLS estimators have the least variance among the class of all linear unbiased estimators. Journal of Statistical Planning and Inference, 88, 173--179. (See text for easy proof). MMSE with linear measurements consider speciﬁc case y = Ax+v, x ∼ N(¯x, ... proof: multiply Restrict estimate to be linear in data x 2. (ii) If a0β is estimable, there is … We are restricting our search for estimators to the class of linear, unbiased ones. Find the best one (i.e.