5.4 Asymptotic confidence intervals

We assume now that the rv \(X\) that represents the population follows a distribution that belongs to a parametric family of distributions \(\{F(\cdot;\theta):\theta\in\Theta\},\) and that we want to obtain a confidence interval for \(\theta.\) This family does not have to be normal and may even be unknown. In this situation, we may have an asymptotic pivot of the form

\[\begin{align} Z(\theta)=\frac{\hat{\theta}-\theta}{\hat{\sigma}(\hat{\theta})}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1).\tag{5.6} \end{align}\]

Then, an asymptotic confidence interval for \(\theta\) is given by

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\theta):=\left[\hat{\theta}\mp z_{\alpha/2}\hat{\sigma}(\hat{\theta})\right]. \end{align*}\]

This confidence interval has a confidence level approximately \(1-\alpha,\) that is

\[\begin{align*} \mathbb{P}(\theta\in\mathrm{ACI}_{1-\alpha}(\theta))\approx 1-\alpha. \end{align*}\]

This approximation becomes more accurate as the sample size \(n\) grows due to (5.6).

5.4.1 Asymptotic confidence interval for the mean

Let \(X\) be a rv with \(\mathbb{E}[X]=\mu\) and \(\mathbb{V}\mathrm{ar}[X]=\sigma^2,\) both being unknown parameters, and let \((X_1,\ldots,X_n)\) be a srs of \(X.\) In Example 3.17 we have seen that

\[\begin{align*} Z(\mu)=\frac{\bar{X}-\mu}{S'/\sqrt{n}}\stackrel{d}{\longrightarrow} \mathcal{N}(0,1). \end{align*}\]

Therefore, an asymptotic confidence interval for \(\mu\) at the confidence level \(1-\alpha\) is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\mu)=\left[\bar{X}\mp z_{\alpha/2}\frac{S'}{\sqrt{n}}\right]. \end{align*}\]

Example 5.8 The shopping times of \(n=64\) random customers from a local supermarket were measured. The sample mean and the quasivariance were \(33\) and \(256\) minutes, respectively. Estimate the average shopping time \(\mu\) of a customer with a \(90\%\) confidence level.

The asymptotic confidence interval for \(\mu\) at significance level \(\alpha=0.10\) is

\[\begin{align*} \mathrm{ACI}_{0.90}(\mu)=\left[\bar{X}\mp z_{0.05}\frac{S'}{\sqrt{n}}\right]\approx\left[33\mp 1.645\sqrt{\frac{256}{64}}\right]\approx[29.71,36.29]. \end{align*}\]

Example 5.9 Let \((X_1,\ldots,X_n)\) be a srs of a rv with distribution \(\mathrm{Pois}(\lambda).\) Let us compute an asymptotic confidence interval at significance level \(\alpha\) for \(\lambda.\)

In Exercise 3.11 we have seen that

\[\begin{align*} \frac{\bar{X}-\lambda}{\sqrt{\bar{X}/n}}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]

Therefore, the asymptotic confidence interval for \(\lambda\) is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\lambda)=\left[\bar{X}\mp z_{\alpha/2} \sqrt{\frac{\bar{X}}{n}}\right]. \end{align*}\]

5.4.2 Asymptotic confidence interval for the proportion

Let \(X_1,\ldots,X_n\) be iid rv’s with \(\mathrm{Ber}(p)\) distribution. We are going to obtain an asymptotic confidence interval at confidence level \(1-\alpha\) for \(p.\)

In Example 3.18 we have seen that

\[\begin{align*} Z(p)=\frac{\hat{p}-p}{\sqrt{\hat{p}(1-\hat{p})/n}}\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]

Therefore, an asymptotic confidence interval for \(p\) is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(p)=\left[\hat{p}\mp z_{\alpha/2}\sqrt{\frac{\hat{p} (1-\hat{p})}{n}}\right]. \end{align*}\]

This asymptotic confidence interval can be easily extended to the two proportions case.

Example 5.10 Two brands of refrigerators, A and B, have a warranty of one year. In a random sample of \(n_A=50\) refrigerators of A, \(12\) failed before the end of the warranty period. From the random sample of \(n_B=60\) refrigerators of B, \(12\) failed before the expiration of the warranty. Estimate the difference of the failure proportions during the warranty period at the confidence level \(0.98.\)

Let \(p_A\) and \(p_B\) be the proportion of failures for A and B, respectively. By the CLT we know that, for a large sample size, the sample proportions \(\hat{p}_A\) and \(\hat{p}_B\) verify

\[\begin{align*} \hat{p}_A\cong\mathcal{N}\left(p_A, \frac{p_A (1-p_A)}{n_A}\right),\quad \hat{p}_B\cong\mathcal{N}\left(p_B,\frac{p_B (1-p_B)}{n_B}\right). \end{align*}\]

Then, for large \(n_A\) and \(n_B\) it is verified

\[\begin{align*} \hat{p}_A-\hat{p}_B\cong\mathcal{N}\left(p_A-p_B,\frac{p_A (1-p_A)}{n_A}+\frac{p_B (1-p_B)}{n_B}\right). \end{align*}\]

Applying Corollary 3.3 and Theorem 3.4, a pivot is

\[\begin{align*} Z(p_A-p_B)=\frac{(\hat{p}_A-\hat{p}_B)-(p_A-p_B)}{\sqrt{{\frac{\hat{p}_A (1-\hat{p}_A)}{n_A}+\frac{\hat{p}_B (1-\hat{p}_B)}{n_B}}}}\stackrel{d}{\longrightarrow} \mathcal{N}(0,1) \end{align*}\]

so an asymptotic confidence interval for \(p_A-p_B\) is

\[\begin{align*} \mathrm{ACI}_{0.98}(p_A-p_B)=\left[(\hat{p}_A-\hat{p}_B)\mp z_{0.01}\sqrt{\frac{\hat{p}_A (1-\hat{p}_A)}{n_A}+\frac{\hat{p}_B (1-\hat{p}_B)}{n_B}}\right]. \end{align*}\]

The sample proportions for the refrigerators are \(\hat{p}_A=12/50=0.24\) and \(\hat{p}_B=12/60=0.20,\) so the above asymptotic confidence interval is

\[\begin{align*} \mathrm{ACI}_{0.98}(p_A-p_B)&\approx\left[(0.24-0.20)\mp 2.33\sqrt{\frac{0.24\times 0.76}{50}+\frac{0.20\times 0.80}{60}}\right]\\ &\approx[-0.145,0.225]. \end{align*}\]

This confidence interval contains zero. This indicates no significant difference between \(p_A\) and \(p_B\) at the 98% confidence level.

5.4.3 Asymptotic maximum likelihood confidence intervals

Theorem 4.1 and Corollary 4.2 provide a readily usable asymptotic pivot for \(\theta\) based on the very general maximum likelihood estimator:

\[\begin{align*} Z(\theta)=\frac{\hat{\theta}_{\mathrm{MLE}}-\theta}{1\big/\sqrt{n\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})}}=\sqrt{n\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})}\left(\hat{\theta}_{\mathrm{MLE}}-\theta\right)\stackrel{d}{\longrightarrow}\mathcal{N}(0,1). \end{align*}\]

The result can be precised as follows.

Theorem 5.1 (Asymptotic maximum likelihood confidence intervals) Let \(X\sim f(\cdot;\theta).\) Under the conditions of Theorem 4.1 and Corollary 4.2, the asymptotic maximum likelihood confidence interval

\[\begin{align} \mathrm{ACI}_{1-\alpha}(\theta)=\left[\hat{\theta}_{\mathrm{MLE}}\mp \frac{z_{\alpha/2}}{\sqrt{n\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})}}\right] \tag{5.7} \end{align}\]

is such that \(\mathbb{P}(\theta \in \mathrm{ACI}_{1-\alpha}(\theta))\to 1 - \alpha\) as \(n\to\infty.\)

Remark. An analogous result can be established for a discrete rv \(X\sim F(\cdot;\theta).\)

Remark. Due to Corollary 4.3, \(\mathcal{I}(\hat{\theta}_{\mathrm{MLE}})\) can be replaced with \(\hat{\mathcal{I}}(\hat{\theta}_{\mathrm{MLE}})\) in (5.7) without altering the asymptotic coverage probability. This is especially useful when \(\mathcal{I}(\theta)\) is difficult to compute, since \(\hat{\mathcal{I}}(\theta)=\frac{1}{n}\sum_{i=1}^n (\partial\log f(X_i;\theta)/\partial\theta)^2\) is straightforward to compute from the srs \((X_1,\ldots,X_n)\) from \(X\sim f(\cdot;\theta).\)

Example 5.11 Let us compute the asymptotic maximum likelihood confidence interval for \(\lambda\) in a \(\mathrm{Exp}(\lambda)\) distribution.

We know from Example 4.12 that \(\hat{\lambda}_{\mathrm{MLE}}=1/{\bar{X}}\) and \(\mathcal{I}(\lambda)=1/\lambda^2.\) Therefore, the asymptotic confidence interval is

\[\begin{align*} \mathrm{ACI}_{1-\alpha}(\lambda)=\left[\hat{\lambda}_{\mathrm{MLE}}\mp z_{\alpha/2}\frac{\hat{\lambda}_{\mathrm{MLE}}}{\sqrt{n}}\right]. \end{align*}\]

To illustrate this confidence interval, let us generate a simulated sample of size \(n=100\) for \(\lambda=2\) and compute the asymptotic \(95\%\)-confidence interval:

# Sample from Exp(2)
set.seed(123456)
n <- 100
x <- rexp(n = n, rate = 2)

# MLE
lambda_mle <- 1 / mean(x)

# Asymptotic confidence interval
alpha <- 0.05
z <- qnorm(alpha / 2, lower.tail = FALSE)
lambda_mle + c(-1, 1) * z * lambda_mle / sqrt(n)
## [1] 1.374268 2.044294

In this case, \(\lambda=2\) belongs to the confidence interval.