A confidence interval, also known as the acceptance region, is a set of values for the test statistic for which the null hypothesis is accepted. i.e. if the observed test statistic is in the confidence interval then we accept the null hypothesis and reject the alternative hypothesis.
Confidence intervals can be calculated at different significance levels. We use $\alpha$ to denote the level of significance and perform a hypothesis test with a $100(1- \alpha)$% confidence interval.
Confidence intervals are usually calculated at $5$% or $1$% significance levels, for which $\alpha = 0.05$ and $\alpha = 0.01$ respectively. Note that a $95$% confidence interval does not mean there is a $95$% chance that the true value being estimated is in the calculated interval. Rather, given a population, there is a $95$% chance that choosing a random sample from this population results in a confidence interval which contains the true value being estimated.
A critical region, also known as the rejection region, is a set of values for the test statistic for which the null hypothesis is rejected. i.e. if the observed test statistic is in the critical region then we reject the null hypothesis and accept the alternative hypothesis.
The critical value at a certain significance level can be thought of as a cut-off point. If a test statistic on one side of the critical value results in accepting the null hypothesis, a test statistic on the other side will result in rejecting the null hypothesis.
Usually, the easiest way to perform a hypothesis test with the binomial distribution is to use the $p$-value and see whether it is larger or smaller than $\alpha$, the significance level used.
Sometimes, if we have observed a large number of Bernoulli Trials, we can use the observed probability of success $\hat{p}$, based entirely on the data obtained, to approximate the distribution of error using the normal distribution. We do this using the formula \[\hat{p} \pm z_{1-\frac{\alpha}{2} } \sqrt{ \frac{1}{n} \hat{p} (1-\hat {p})}\] where $\hat{p}$ is the estimated probability of success, $z_{1- \frac{\alpha}{2} }$ is obtained from the normal distribution tables, $\alpha$ is the significance level and $n$ is the sample size.
A coin is tossed $1050$ times and lands on heads $500$ times. Construct a $90$% confidence interval for the probability $p$ of getting a head.
Here the observed probability of success $\hat{p} = \dfrac{500}{1050}$, $n=1050$ and $\alpha = 0.1$ so $z_{1-\frac{\alpha}{2} } = z_{0.95} = 1.645$. This is because $\Phi^{-1} (0.95) = 1.645$.
So the confidence interval will be between $\hat{p} + z_{1-\frac{\alpha}{2} } \sqrt{ \frac{1}{n} \hat{p} (1-\hat {p})} \text{ and } \hat{p} - z_{1-\frac{\alpha}{2} } \sqrt{ \frac{1}{n} \hat{p} (1-\hat {p})} . $ By substituting into these expressions, we find that the confidence interval is between \begin{align} &\dfrac{500}{1050} + 1.645 \sqrt{ \frac{1}{1050} \times \dfrac{500}{1050} \times \left(1- \dfrac{500}{1050}\right) }\\ \text{ and } &\dfrac{500}{1050} - 1.645 \sqrt{ \frac{1}{1050} \times \dfrac{500}{1050} \times \left(1- \dfrac{500}{1050}\right) }\\\\ &=0.47619 + (1.645 \times \sqrt{0.00024} ) \text{ and } 0.47619 - (1.645 \times \sqrt{0.00024} ) \\ &=0.50155 \text{ and } 0.45084 . \end{align} So the confidence interval is $(0.45084, 0.50155)$.
We can use either the $z$-score or the sample mean $\bar{x}$ as the test statistic. If the $z$-score is used then reading straight from the tables gives the critical values.
For example, the critical values for a $5$% significance test are:
To obtain a confidence interval for the mean, use the following procedure:
For a two-tailed test with a $5$% significance level we need to consider \begin{align} 0.95 &= \mathrm{P}[-k< Z < k] \\ &= \mathrm{P}\left[-k<\dfrac{\bar{X}-\mu}{\frac{\sigma}{\sqrt{n} } }
Given the number of degrees of freedom $v$ and the significance level $\alpha$, the critical values can be obtained from the tables. Critical regions can then be computed from these.
If we are performing a hypothesis test at a $1$% significance level with $15$ degrees of freedom using the Student $t$-distribution then there are three cases, depending on the alternative hypothesis.
If we are performing a two-tailed test, the critical values are $\pm2.9467$ so the confidence interval is $-2.9467 \leq t \leq 2.9467$ where $t$ is the test statistic. The critical regions will be $t< -2.9467$ and $t>2.9467$.
If we are performing a one-tailed test, the critical value is $2.6025$:
In this video, Daniel Organisciak calculates a one-tailed confidence interval for the normal distribution.
In this video Daniel Organisciak calculates a two-tailed confidence interval for the normal distribution.