Exact Estimation vs. Approximate Estimation
Most frequentist inference utilizes approximate estimation rather than exact estimation like Bayesian inference. Exact estimation is based on exact probability statements that are valid for any sample size. In contrast, approximate estimation is based in asymptotic theory. In math, an asymptote is a real-valued function with a curve that gets closer to the axis as the value along the axis approaches infinity, though it will never "touch" the axis. Consider the inverse of x, which is 1/x.. No matter how large x becomes, the result will never equal zero.
Approximate estimates are often unstable with small sample sizes. The Bayesian use of exact estimates helps avoid some of the unreasonable assumptions of traditional frequentist inference, such as by quantifying uncertainty with small sample sizes, or obviating the assumption of equal variances in ANOVA (Analysis of Variance).
Most frequentist inferences consist of maximum likelihood estimates, which rely on normality approximations based on large sample asymptotics. Maximum likelihood does not obey the likelihood principle, but that's another topic. Most Bayesian inferences consist of one or more forms of Markov chain Monte Carlo (MCMC) estimates, which provide exact probabilities, provided the chain or chains have converged.
Approximate estimation is simpler. Exact estimation is better.