Unbiasedness

An unbiased estimator is an estimator whose expected value is equal to the true value of the population parameter being estimated. Unbiasedness is one of the most important finite-sample properties used to evaluate statistical estimators.

It indicates that an estimation procedure does not systematically overestimate or underestimate the unknown parameter. However, unbiasedness does not mean that an estimate obtained from every individual sample will be exactly equal to the true parameter value.

Definition of an Unbiased Estimator

Let $\theta$ be an unknown population parameter and let $T=T(X_1,X_2,\ldots,X_n)$ be an estimator of $\theta$ . The estimator $T$ is said to be an unbiased estimator of $\theta$ if

$E_{\theta}(T)=\theta$

for every possible value of $\theta$ in the parameter space.

The notation $E_{\theta}(T)$ emphasizes that the expectation is calculated under the probability distribution determined by the parameter $\theta$ .

If repeated random samples of the same size are drawn from the population and $T$ is calculated from each sample, the average of all the resulting estimates will approach the true value $\theta$ .

Bias of an Estimator

The bias of an estimator $T$ for the parameter $\theta$ is defined as

$\operatorname{Bias}_{\theta}(T)=E_{\theta}(T)-\theta$

Therefore, $T$ is unbiased for $\theta$ when

$\operatorname{Bias}_{\theta}(T)=0$

The following cases are possible:

If $E_{\theta}(T)=\theta$ , then $T$ is unbiased.
If $E_{\theta}(T)>\theta$ , then $T$ has positive bias and tends to overestimate $\theta$ .
If $E_{\theta}(T)<\theta$ , then $T$ has negative bias and tends to underestimate $\theta$ .

Example: Sample Mean as an Unbiased Estimator

Let $X_1,X_2,\ldots,X_n$ be a random sample from a population having mean $\mu$ and variance $\sigma^2$ . The sample mean is

$\bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_i$

Taking expectation and using the linearity of expectation,

$E(\bar{X})=E\left(\frac{1}{n}\sum_{i=1}^{n}X_i\right)$

$=\frac{1}{n}\sum_{i=1}^{n}E(X_i)$

Since $E(X_i)=\mu$ for every $i$ ,

$E(\bar{X})=\frac{1}{n}\sum_{i=1}^{n}\mu=\frac{n\mu}{n}=\mu$

Thus,

$E(\bar{X})=\mu$

Therefore, the sample mean $\bar{X}$ is an unbiased estimator of the population mean $\mu$ .

Example: Sample Proportion

Suppose $X$ is the number of successes in $n$ independent Bernoulli trials, each having probability of success $p$ . Then

$X\sim\operatorname{Binomial}(n,p)$

The sample proportion is

$\hat{p}=\frac{X}{n}$

Since $E(X)=np$ ,

$E(\hat{p})=E\left(\frac{X}{n}\right)=\frac{1}{n}E(X)$

$=\frac{np}{n}=p$

Hence, $\hat{p}$ is an unbiased estimator of the population proportion $p$ .

Example: Unbiased Estimation of Population Variance

Consider the estimator

$S_n^2=\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2$

Its expected value is

$E(S_n^2)=\frac{n-1}{n}\sigma^2$

Since this is not equal to $\sigma^2$ , the estimator $S_n^2$ is biased. Its bias is

$\operatorname{Bias}(S_n^2)=\frac{n-1}{n}\sigma^2-\sigma^2$

$=-\frac{\sigma^2}{n}$

An unbiased estimator of the population variance is obtained by replacing $n$ in the denominator with $n-1$ :

$S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2$

For this estimator,

$E(S^2)=\sigma^2$

Therefore, $S^2$ is an unbiased estimator of $\sigma^2$ .

Example: Correcting a Biased Estimator

Let $X_1,X_2,\ldots,X_n$ be a random sample from the uniform distribution on $(0,\theta)$ , and let

$X_{(n)}=\max(X_1,X_2,\ldots,X_n)$

The expected value of the sample maximum is

$E(X_{(n)})=\frac{n}{n+1}\theta$

Thus, $X_{(n)}$ is a biased estimator of $\theta$ . Multiplying it by $\frac{n+1}{n}$ gives

$T=\frac{n+1}{n}X_{(n)}$

Now,

$E(T)=\frac{n+1}{n}E(X_{(n)})$

$=\frac{n+1}{n}\times\frac{n}{n+1}\theta=\theta$

Therefore, $\frac{n+1}{n}X_{(n)}$ is an unbiased estimator of $\theta$ .

Properties of Unbiased Estimators

1. Unbiasedness Is Defined with Respect to a Parameter

An estimator is not simply called unbiased without specifying the quantity it estimates. An estimator may be unbiased for one function of a parameter but biased for another function.

For example, if $T$ is unbiased for $\theta$ , then

$E(T)=\theta$

However, this does not necessarily mean that $T^2$ is unbiased for $\theta^2$ .

2. An Unbiased Estimator Need Not Be Unique

There may be several unbiased estimators of the same parameter. For example, if $X_1,X_2,\ldots,X_n$ are identically distributed with mean $\mu$ , then each individual observation $X_i$ is unbiased for $\mu$ .

Also, the sample mean $\bar{X}$ is unbiased for $\mu$ . Therefore, both $X_1$ and $\bar{X}$ are unbiased estimators of $\mu$ , although their variances are generally different.

3. Linear Transformation Property

If $T$ is an unbiased estimator of $\theta$ , then for constants $a$ and $b$ , the estimator $aT+b$ is unbiased for $a\theta+b$ .

Since

$E(aT+b)=aE(T)+b$

and $E(T)=\theta$ ,

$E(aT+b)=a\theta+b$

Thus, $aT+b$ is unbiased for $a\theta+b$ .

4. Linear Combination of Unbiased Estimators

Suppose $T_1,T_2,\ldots,T_k$ are unbiased estimators of $\theta_1,\theta_2,\ldots,\theta_k$ , respectively. For constants $a_1,a_2,\ldots,a_k$ ,

$E\left(\sum_{i=1}^{k}a_iT_i\right)=\sum_{i=1}^{k}a_iE(T_i)$

$=\sum_{i=1}^{k}a_i\theta_i$

Therefore,

$\sum_{i=1}^{k}a_iT_i$

is an unbiased estimator of

$\sum_{i=1}^{k}a_i\theta_i$

In particular, if $T_1,T_2,\ldots,T_k$ are all unbiased for the same parameter $\theta$ and

$\sum_{i=1}^{k}a_i=1$

then

$\sum_{i=1}^{k}a_iT_i$

is also unbiased for $\theta$ .

5. Nonlinear Transformations Do Not Generally Preserve Unbiasedness

If $T$ is unbiased for $\theta$ , then $g(T)$ is not necessarily unbiased for $g(\theta)$ . In general,

$E[g(T)]\neq g[E(T)]$

For example, if $T$ is unbiased for $\theta$ , then

$E(T)=\theta$

But

$E(T^2)=\operatorname{Var}(T)+[E(T)]^2$

$=\operatorname{Var}(T)+\theta^2$

Therefore, unless $\operatorname{Var}(T)=0$ ,

$E(T^2)\neq\theta^2$

Hence, $T^2$ is generally not an unbiased estimator of $\theta^2$ .

6. Unbiasedness Does Not Guarantee Minimum Variance

Two estimators may both be unbiased for the same parameter but may have different variances. The estimator with the smaller variance gives estimates that are more concentrated around the true parameter value.

Suppose $T_1$ and $T_2$ are unbiased for $\theta$ . If

$\operatorname{Var}(T_1)<\operatorname{Var}(T_2)$

then $T_1$ is considered more efficient than $T_2$ .

7. Unbiasedness Does Not Guarantee Consistency

An estimator can be unbiased for every sample size and still fail to be consistent. Consistency requires the estimator to converge in probability to the true parameter as the sample size increases:

$T_n\xrightarrow{P}\theta$

An unbiased estimator $T_n$ will be consistent if its variance approaches zero:

$\operatorname{Var}(T_n)\to 0\quad\text{as}\quad n\to\infty$

Therefore, unbiasedness alone is not sufficient for consistency.

8. A Biased Estimator Can Be Consistent

An estimator may be biased for every finite sample size but still be consistent if its bias and variance both approach zero as the sample size increases.

For example,

$S_n^2=\frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2$

has bias

$-\frac{\sigma^2}{n}$

Since

$-\frac{\sigma^2}{n}\to 0$

as $n\to\infty$ , the estimator is asymptotically unbiased and consistent under standard conditions.

9. Unbiasedness and Mean Squared Error

The mean squared error of an estimator $T$ is

$\operatorname{MSE}(T)=E\left[(T-\theta)^2\right]$

It can be decomposed as

$\operatorname{MSE}(T)=\operatorname{Var}(T)+[\operatorname{Bias}(T)]^2$

If $T$ is unbiased, then

$\operatorname{Bias}(T)=0$

and therefore,

$\operatorname{MSE}(T)=\operatorname{Var}(T)$

Thus, among unbiased estimators, minimizing the mean squared error is equivalent to minimizing the variance.

10. Unbiasedness Is a Finite-Sample Property

Unbiasedness is usually examined for a fixed sample size. If

$E(T_n)=\theta$

for every $n$ , then $T_n$ is unbiased for every sample size.

By contrast, asymptotic unbiasedness only requires

$\lim_{n\to\infty}E(T_n)=\theta$

An asymptotically unbiased estimator may remain biased for every finite value of $n$ .

11. Unbiased Estimators May Not Exist

For some statistical models and some functions of parameters, no unbiased estimator exists. The existence of an unbiased estimator depends on the sampling distribution and the function of the parameter being estimated.

Therefore, it should not be assumed that every parameter automatically has an unbiased estimator.

12. Rao-Blackwell Improvement Property

Suppose $T$ is an unbiased estimator of $\theta$ and $S$ is a sufficient statistic for $\theta$ . Define

$T^*=E(T\mid S)$

Then $T^*$ is also unbiased for $\theta$ , because

$E(T^*)=E[E(T\mid S)]=E(T)=\theta$

Moreover,

$\operatorname{Var}(T^*)\leq\operatorname{Var}(T)$

Thus, conditioning an unbiased estimator on a sufficient statistic produces an unbiased estimator with variance no greater than that of the original estimator.

13. Completeness and Uniqueness of an Unbiased Estimator

If $S$ is a complete sufficient statistic for $\theta$ , then any unbiased estimator that is a function of $S$ is unique almost surely.

This result is a consequence of the Lehmann-Scheffe theorem. It also implies that such an estimator is the uniformly minimum variance unbiased estimator of the parameter.

Minimum Variance Unbiased Estimator

An estimator $T$ is called a minimum variance unbiased estimator of $\theta$ if it satisfies the following conditions:

$T$ is unbiased for $\theta$ .
Its variance is less than or equal to the variance of every other unbiased estimator of $\theta$ .

If the variance is minimum for every possible value of $\theta$ , then $T$ is called a uniformly minimum variance unbiased estimator, abbreviated as UMVUE.

Formally, an unbiased estimator $T^*$ is a UMVUE of $\theta$ if

$\operatorname{Var}_{\theta}(T^*)\leq\operatorname{Var}_{\theta}(T)$

for every unbiased estimator $T$ of $\theta$ and for every value of $\theta$ .

Unbiasedness and Efficiency

Suppose $T_1$ and $T_2$ are two unbiased estimators of the same parameter $\theta$ . Their relative efficiency may be defined as

$\operatorname{Eff}(T_1,T_2)=\frac{\operatorname{Var}(T_2)}{\operatorname{Var}(T_1)}$

$\operatorname{Eff}(T_1,T_2)>1$

then $T_1$ has a smaller variance and is more efficient than $T_2$ .

Unbiasedness Versus Consistency

Unbiasedness	Consistency
It requires $E(T_n)=\theta$ .	It requires $T_n\xrightarrow{P}\theta$ .
It is generally a finite-sample property.	It is a large-sample property.
It concerns the average value of the estimator.	It concerns convergence to the true parameter.
An unbiased estimator need not be consistent.	A consistent estimator need not be unbiased for finite samples.

Unbiasedness Versus Sufficiency

Unbiasedness and sufficiency describe different features of an estimator or statistic.

Unbiasedness concerns whether the expected value equals the parameter.
Sufficiency concerns whether a statistic contains all the sample information about the parameter.

An unbiased estimator need not be sufficient, and a sufficient statistic need not be an unbiased estimator of a particular parameter. However, sufficient statistics are useful for improving unbiased estimators through the Rao-Blackwell theorem.

Is an Unbiased Estimator Always the Best Estimator?

An unbiased estimator is not necessarily the best estimator in every practical situation. It may have a very large variance and produce highly unstable estimates.

A slightly biased estimator may have a much smaller variance. Since

$\operatorname{MSE}(T)=\operatorname{Var}(T)+[\operatorname{Bias}(T)]^2$

a biased estimator can have a smaller mean squared error than an unbiased estimator if the reduction in variance is sufficiently large.

Therefore, unbiasedness should be considered together with variance, mean squared error, consistency, efficiency and sufficiency when evaluating an estimator.

How to Find an Unbiased Estimator

A general procedure for finding an unbiased estimator is:

Choose a statistic $T$ that depends only on the sample observations.
Calculate its expected value $E(T)$ .
Express $E(T)$ in terms of the unknown parameter.
Modify $T$ , if necessary, so that its expected value equals the required parameter.

For example, suppose

$E(T)=a\theta+b$

where $a\neq 0$ and $a$ and $b$ are known constants. Define

$T^*=\frac{T-b}{a}$

Then

$E(T^*)=E\left(\frac{T-b}{a}\right)$

$=\frac{E(T)-b}{a}$

$=\frac{a\theta+b-b}{a}=\theta$

Therefore, $T^*$ is an unbiased estimator of $\theta$ .

Important Points to Remember

An estimator $T$ is unbiased for $\theta$ if $E(T)=\theta$ .
The bias of an estimator is $E(T)-\theta$ .
Unbiasedness means correctness on average over repeated samples.
An unbiased estimator does not necessarily give the true parameter value for every sample.
More than one unbiased estimator may exist for the same parameter.
A linear combination of unbiased estimators is unbiased for the corresponding linear combination of parameters.
Nonlinear transformations do not generally preserve unbiasedness.
An unbiased estimator may have a large variance.
Among unbiased estimators, the one with the smallest variance is preferred.
Unbiasedness does not automatically imply consistency, efficiency or sufficiency.
A biased estimator may have a smaller mean squared error than an unbiased estimator.
A function of a complete sufficient statistic that is unbiased is the unique UMVUE.

Conclusion

An unbiased estimator is an estimator whose expected value equals the true population parameter. It provides a procedure that is correct on average over repeated sampling. Common examples include the sample mean as an estimator of the population mean, the sample proportion as an estimator of a population proportion and the corrected sample variance as an estimator of the population variance.

Although unbiasedness is an important and desirable property, it is not sufficient by itself to identify the best estimator. The variance, mean squared error, consistency, efficiency and sufficiency of the estimator should also be considered.

An unbiased estimator does not guarantee a perfect estimate from every sample; it guarantees that the estimation method is correct on average.

Unbiasedness

Definition of an Unbiased Estimator

Bias of an Estimator

Example: Sample Mean as an Unbiased Estimator

Example: Sample Proportion

Example: Unbiased Estimation of Population Variance

Example: Correcting a Biased Estimator

Properties of Unbiased Estimators

1. Unbiasedness Is Defined with Respect to a Parameter

2. An Unbiased Estimator Need Not Be Unique

3. Linear Transformation Property

4. Linear Combination of Unbiased Estimators

5. Nonlinear Transformations Do Not Generally Preserve Unbiasedness

6. Unbiasedness Does Not Guarantee Minimum Variance

7. Unbiasedness Does Not Guarantee Consistency

8. A Biased Estimator Can Be Consistent

9. Unbiasedness and Mean Squared Error

10. Unbiasedness Is a Finite-Sample Property

11. Unbiased Estimators May Not Exist

12. Rao-Blackwell Improvement Property

13. Completeness and Uniqueness of an Unbiased Estimator

Minimum Variance Unbiased Estimator

Unbiasedness and Efficiency

Unbiasedness Versus Consistency

Unbiasedness Versus Sufficiency

Is an Unbiased Estimator Always the Best Estimator?

How to Find an Unbiased Estimator

Important Points to Remember

Conclusion

Leave a Comment Cancel reply

Contact Us

For Job Seekers

For Employers

Company

Login to JobOrBlog

Reset Password

Create a free JobOrBlog account

Unbiasedness

Definition of an Unbiased Estimator

Bias of an Estimator

Example: Sample Mean as an Unbiased Estimator

Example: Sample Proportion

Example: Unbiased Estimation of Population Variance

Example: Correcting a Biased Estimator

Properties of Unbiased Estimators

1. Unbiasedness Is Defined with Respect to a Parameter

2. An Unbiased Estimator Need Not Be Unique

3. Linear Transformation Property

4. Linear Combination of Unbiased Estimators

5. Nonlinear Transformations Do Not Generally Preserve Unbiasedness

6. Unbiasedness Does Not Guarantee Minimum Variance

7. Unbiasedness Does Not Guarantee Consistency

8. A Biased Estimator Can Be Consistent

9. Unbiasedness and Mean Squared Error

10. Unbiasedness Is a Finite-Sample Property

11. Unbiased Estimators May Not Exist

12. Rao-Blackwell Improvement Property

13. Completeness and Uniqueness of an Unbiased Estimator

Minimum Variance Unbiased Estimator

Unbiasedness and Efficiency

Unbiasedness Versus Consistency

Unbiasedness Versus Sufficiency

Is an Unbiased Estimator Always the Best Estimator?

How to Find an Unbiased Estimator

Important Points to Remember

Conclusion

Share this post

Leave a Comment Cancel reply

Recent News Articles

Asymptotic Relative Efficiency

Consistent Estimator

Contact Us

For Job Seekers

For Employers

Company