Unbiasedness and small variance are desirable properties of estimators

Question

However, you can imagine situations where a trade-off exists between the two: one estimator may be have a small bias but a much smaller variance than another, unbiased estimator. The concept of "mean square error" estimator combines the two concepts. Let be an estimator of ?. Then the mean square error (MSE) is defined as follows: MSE( ) = E( – ?)2. Prove that MSE( ) = bias2 + var( ). (Hint: subtract and add in E( ) in E( – ?)2.)
What will be an ideal response?

Answer 1

Answer:
MSE ( ) = E( - E( ) + E( ) - μ)2 = E[( - E( )) + (E( ) - μ)]2
= E[( - E( ))2 + (E( ) - μ)2 + 2( - E( ))(E( ) - μ)]
Next, moving through the expectation operator results in
E[ - E( )]2 + E[E( ) - μ)]2 + 2E[( ) - E( ))( E( ) - μ)].
The first term is the variance, and the second term is the squared bias, since
E[E( ) - μ)]2 = [E( ) - μ)]2. This proves MSE ( ) = bias2 + var( ) if the last term equals zero. But
E[( - E( ))(E( ) - μ)] = E[E( ) - μ - (E( ))2 + μE( )]
= E( ) E( ) - μE( ) - (E( ))2 + μE( ) = 0.