What does the significance level $\alpha$ control in hypothesis testing?

In a symmetric two-tailed construction, the significance level $\alpha$ is split equally between the two tails. Each tail receives $\alpha/2$.

Conditions: The test is a symmetric two-tailed test.; The significance level is $\alpha$.

The form of the alternative hypothesis dictates the tail(s) of the test. $H_1:\mu\ne23$ gives a two-tailed test, $H_1:\mu<23$ a left-tailed test, and $H_1:\mu>23$ a right-tailed test.

Conditions: The null hypothesis is based on a population mean of 23.; The alternative hypothesis is formulated before seeing the data.

Failing to reject the null hypothesis means that the sample provides insufficient evidence against it. It is not a proof that the null is true; it simply indicates that the data do not contradict the null hypothesis strongly enough to warrant rejection.

Conditions: A hypothesis test has been conducted.; The decision is to fail to reject $H_0$.

The null hypothesis includes the boundary value. In the example with a population mean of 23, the null hypothesis is centered around or includes this specific value as the reference claim against which the alternative hypothesis tests for departure.

Conditions: The example uses a population mean of 23.; The null hypothesis is formulated to include this value.

The null hypothesis serves as the reference claim that includes the boundary value. In the example, it represents the status quo or the specific departure being tested against the alternative hypothesis.

Conditions: The population mean is 23.; The null hypothesis includes the boundary value.