STATISTICAT

Sample Size

Sample size is the number of observations included in a statistical model. Typically, a larger sample size allows more precision in the estimates of a statistical model. Frequentist inference has a potential problem with sample sizes that are too large, and too small. When sample size is too large in frequentist inference, hypothesis testing becomes biased in favor of rejecting the null hypothesis. When sample size is too small, the frequentist reliance on asymptotic approximations becomes inappropriate, and the estimates become biased.

In contrast, Bayesian inference does not force an artificial dichotomy between a null and alternative hypothesis, allows exact probability statements about any hypothesis, and is not biased due to a large sample size. Regarding small sample size, Bayesian inference is also unbiased due to exact estimation (as opposed to asymptotic approximation), and still permits exact probability statements. For example, it is valid in Bayesian inference to design a statistical model with only eight observations. Bayesian inference handles uncertainty better than frequentist inference, and the credible intervals of the parameters will be appropriately wide. If this model is hierarchical, then the statistician may estimate more parameters than observations, which is also valid in Bayesian inference, and invalid in frequentist inference.

Since sample size is a concern with any statistical model, Bayesian inference is preferable to frequentist inference.