Contents

1. Point estimation
2. Interval estimation
3. Confidence Interval

Estimation

Estimation is divided into point estimation, which estimates a specific value of a parameter, and interval estimation, which estimates a certain interval in which the parameter is included.

Point estimation

A statistic used to estimate the parameters of a population from a sample is called an estimator, and an estimate that meets some assumptions and has no bias is used as an unbiased estimator. For example, to estimate the population mean, the sample mean is used as an unbiased estimate.

The sample mean of n samples X₁, X₂, …, X_n is calculated as in Equation 1 and is an estimator of the population mean.

$$\begin{align}\tag{1} \hat{\mu}&=\bar{X}\\ &=\frac{1}{n}(\bar{X_1}+\bar{X_2}+\cdots+\bar{X_n})\\ &=\frac{1}{n}\sum^n_{i=1}\bar{X_i } \end{align}$$

Also, in general, the population variance σ² is an unknown value, so it is calculated as in Equation 2 using the standard deviation s of the sampling distribution as an estimator.

$$\begin{align} \tag{2}\hat{\sigma}&=s\\ &=\sqrt{\frac{1}{n-1}\sum^n_{i=1}(\bar{X_i}-\bar{X})}\\ n :&\text{sample size} \end{align}$$

Interval estimation

Parameters estimated by the statistic of a sample have uncertainty because they vary depending on the sample being sampled from the population. Therefore, rather than representing a parameter with a single value, such as a point estimate, it would be more reasonable to represent an interval that includes the parameter at a level that is probabilistically reliable. These intervals are called confidence intervals.

Confidence Interval

The sample mean ($\bar{x}$) can be a good estimator for the population mean ($\mu$), but uncertainty about perfect agreement exists. Therefore, based on the estimator, it is possible to indicate by setting the interval where the population mean is most likely to exist. For example, assuming a normal distribution, you can specify an interval in which the estimate can exist in either or both directions around the mean with the highest probability. Such an interval is called confidence interval.

If the sample mean is within the confidence interval for estimating the population mean, then the basis for using the sample mean as the population mean is prepared. However, if it is located outside that interval, it may be difficult to use it as a population mean. That is, the assumption of the distribution and the establishment of a confidence interval are used as criteria for accepting or rejecting the estimator.

As shown in Figure 1, the 95% probability interval in the standard normal distribution exists in the interval (μ-1.96σ, μ+1.96 σ). By applying this interval, standardized values as in Equation 3 can be returned to their original values by Equation 4.

$$\begin{equation} \tag{4} Z=\frac{X-\mu_x}{\sigma_x} \end{equation}$$

import numpy as np
import pandas as pd 
import matplotlib.pyplot as plt
from scipy import stats

plt.figure(figsize=(6,3))
x=np.linspace(-3, 3.01, 1000)
y=[stats.norm.pdf(i) for i in x]
plt.plot(x, y, label='N(0, 1)')
plt.axhline(0, color="black")
plt.axvline(0, linestyle="--", color="black", label="Mean", alpha=0.3)
plt.axvline(-1.96, linestyle="--", color="green", label="Lower")
plt.axvline(1.96, linestyle="--", color="red", label="Upper")
plt.fill_between(x, 0, y, where=(x <=-1.96) | (x>=1.95), facecolor="skyblue", alpha=0.5)
plt.text(0, -0.09,"x", size="13", weight="bold")
plt.ylabel("pdf", size="13", weight="bold")
plt.legend(loc="best")
plt.xticks([])
plt.text(-0, -0.05, 0, size="13", weight="bold")
plt.text(-2.4, -0.05, -1.96, size="13", weight="bold")
plt.text(1.6, -0.05, 1.96, size="13", weight="bold")
plt.text(-2.6, 0.02, r"$\mathbf{\frac{\alpha}{2}}$", size="14", weight="bold")
plt.text(-0.3, 0.1, r"1-$\mathbf{\alpha}$(0.9)", size="14", weight="bold")
plt.text(2.0, 0.02, r"$\mathbf{\frac{\alpha}{2}}$", size="14", weight="bold")
plt.show()

Figure 1. Confidence and rejection intervals in the standard normal distribution(α=0.05).

$$\begin{align}\tag{4} &P(-1.96 \le Z \le 1.96)=0.95\\ &\rightarrow P\left(-1.96 \le \frac{\overline{y}-\mu}{\frac{\sigma}{\sqrt{n}}} \le 1.96 \right)=0.95\\ &\rightarrow P\left(\overline{y}-1.96\frac{\sigma}{\sqrt{n}} \le Z \le \overline{y}+1.96\frac{\sigma}{\sqrt{n}}\right)=0.95 \end{align}$$

In the result of Equation 4, based on the average $\mu$, the value on the left is lower bound, and the value on the right is upper bound. The bounds of this confidence interval can be calculated using Equation 5.

$$\begin{align}\tag{5} &\text{CI}_\mu=\overline{y} \pm z_\frac{\alpha}{2}\frac{\alpha}{\sqrt{n}}\\ &n: \text{ssample size}\\ &z_\frac{\alpha}{2}:\text{Standard score corresponding to} \,P=100(1-\alpha)\% \end{align}$$

For example, in Figure 1, the standard normal distribution, the tail of the curve is the region corresponding to the significance level. Conversely, the part excluding the significance level from the overall probability, that is, 1-α, is called confidence level or confidence coefficient.

For a significance level of 0.05, the left and right edges of the curve are 0.025 and 1-0.025. The value corresponding to this point is the standard score and is expressed as z_α/2. This value can be checked using the scipy.stats.norm.ppf(q, loc=0, scale=1) method. This method returns a value corresponding to probability q (0.025 or 0.975). This result is equal to the lower or upper bound returned by the interval(1-α, loc=0, scale=1) method. (The method interval() assumes a two-sided test.(see Hypothesis Test)

round(stats.norm.ppf(0.975), 4), round(stats.norm.ppf(0.025),4)

(1.96, -1.96)

np.around(stats.norm.interval(0.95), 4)

array([-1.96,  1.96])