Asymptotic Relative Efficiency

Introduction

In statistical inference, it is often necessary to compare two estimators of the same population parameter. While finite-sample properties such as unbiasedness and minimum variance are important, statisticians are also interested in how estimators perform when the sample size becomes very large. One of the most useful measures for comparing the long-run performance of estimators is Asymptotic Relative Efficiency (ARE).

Asymptotic Relative Efficiency compares the asymptotic variances of two consistent estimators. The estimator with the smaller asymptotic variance is considered more efficient because it provides more precise estimates for large sample sizes.

Definition of Asymptotic Relative Efficiency

Suppose T1 and T2 are two consistent estimators of the same parameter θ. If their asymptotic variances exist, then the Asymptotic Relative Efficiency (ARE) of T1 relative to T2 is defined as

ARE(T_1,T_2)=\lim_{n\to\infty}\frac{Var(T_2)}{Var(T_1)}

provided the limit exists.

More generally, if

nVar(T_1)\rightarrow V_1 \quad \text{and} \quad nVar(T_2)\rightarrow V_2

then

ARE(T_1,T_2)=\frac{V_2}{V_1}

Interpretation

  • If ARE = 1, both estimators are equally efficient for large samples.
  • If ARE > 1, T1 is asymptotically more efficient than T2.
  • If ARE < 1, T2 is asymptotically more efficient than T1.

An estimator with higher asymptotic relative efficiency requires fewer observations to achieve approximately the same level of precision.

Why Asymptotic Relative Efficiency is Important

Many estimators become approximately normally distributed as the sample size increases. In such situations, comparing their asymptotic variances provides a convenient way to determine which estimator performs better in large samples.

ARE is widely used in:

  • Statistical estimation theory.
  • Hypothesis testing.
  • Maximum likelihood estimation.
  • Non-parametric statistics.
  • Robust statistics.
  • Econometrics and machine learning.

Derivation Using Asymptotic Variance

Suppose two estimators satisfy

\sqrt{n}(T_1-\theta)\xrightarrow{d}N(0,V_1)

and

\sqrt{n}(T_2-\theta)\xrightarrow{d}N(0,V_2)

Then, for sufficiently large n,

Var(T_1)\approx\frac{V_1}{n}

and

Var(T_2)\approx\frac{V_2}{n}

Hence,

ARE(T_1,T_2)=\frac{V_2}{V_1}

Thus, ARE compares only the limiting variances of the estimators.

Example 1: Sample Mean versus Sample Median

Suppose observations are drawn from a normal distribution.

The asymptotic variance of the sample mean is

Var(\bar{X})\approx\frac{\sigma^2}{n}

The asymptotic variance of the sample median is

Var(M)\approx\frac{\pi\sigma^2}{2n}

Therefore,

ARE(\bar{X},M)=\frac{\pi\sigma^2/(2n)}{\sigma^2/n}=\frac{\pi}{2}\approx1.57

Equivalently, the efficiency of the sample median relative to the sample mean is

ARE(M,\bar{X})=\frac{2}{\pi}\approx0.637

This means that under normality, the sample median has approximately 63.7% of the efficiency of the sample mean.

Example 2: Maximum Likelihood Estimator

Under suitable regularity conditions, the Maximum Likelihood Estimator (MLE) satisfies

Var(\hat{\theta}_{MLE})\approx\frac{1}{nI(\theta)}

where I(θ) is the Fisher Information.

If another consistent estimator has asymptotic variance

\frac{c}{n}

then

ARE(\hat{\theta}_{MLE},T)=cI(\theta)

Since the MLE achieves the Cramér-Rao lower bound asymptotically, it is asymptotically efficient under regularity conditions.

Relationship with Fisher Information

If an estimator satisfies

Var(T)\approx\frac{1}{nI(\theta)}

then it is said to be asymptotically efficient.

No regular estimator can have an asymptotic variance smaller than

\frac{1}{nI(\theta)}

Therefore, Fisher Information provides the theoretical benchmark for asymptotic efficiency.

Properties of Asymptotic Relative Efficiency

  • It compares estimators only for large sample sizes.
  • It depends on asymptotic variances.
  • It is independent of finite-sample performance.
  • Higher ARE indicates better long-run precision.
  • It is useful when exact variances are difficult to calculate.
  • It helps compare parametric and non-parametric estimators.

Advantages

  • Simple method for comparing consistent estimators.
  • Useful when sample sizes are large.
  • Widely applicable in theoretical statistics.
  • Provides insight into long-run estimator performance.
  • Frequently used in advanced statistical inference.

Limitations

  • Applies only to asymptotic behaviour.
  • May not accurately describe performance for small samples.
  • Requires asymptotic distributions or asymptotic variances to exist.
  • Different estimators may have similar ARE but different finite-sample properties.

Difference Between Efficiency and Asymptotic Relative Efficiency

Efficiency Asymptotic Relative Efficiency
Usually refers to finite-sample performance. Refers to large-sample performance.
Uses exact variances. Uses asymptotic variances.
Applicable when exact distributions are known. Applicable when only limiting distributions are available.
Depends on the actual sample size. Depends on behaviour as the sample size approaches infinity.

Applications

  • Comparing estimators in estimation theory.
  • Selecting between competing statistical procedures.
  • Evaluating robust estimators.
  • Comparing parametric and non-parametric methods.
  • Designing efficient statistical experiments.
  • Large-sample econometric analysis.

Key Points to Remember

  • Asymptotic Relative Efficiency compares the asymptotic variances of two consistent estimators.
  • An estimator with smaller asymptotic variance is more efficient.
  • If ARE is greater than 1, the first estimator is asymptotically more efficient.
  • If ARE equals 1, both estimators have the same asymptotic efficiency.
  • Maximum Likelihood Estimators are asymptotically efficient under regularity conditions.
  • ARE is one of the most important tools for comparing estimators in large-sample statistical inference.

Conclusion

Asymptotic Relative Efficiency is an essential concept in estimation theory because it measures the comparative performance of consistent estimators when the sample size becomes very large. Rather than focusing on finite-sample behaviour, it evaluates long-run precision through asymptotic variances. Since many practical statistical methods rely on large datasets, ARE plays a crucial role in selecting efficient estimators and understanding the theoretical properties of statistical procedures.

inference

Leave a Comment

Recent News Articles

Fresh job related news content posted each day.