Bayesian Methods on Home

Bayesian Classifiation by Mixture Models

Mon, 01 Jan 0001 00:00:00 +0000

Before discussing Bayesian classifcation methods we intoduce the concept of mixture models, often used for unsupervised classification tasks. Mixture models are a diverse set of statistical models which can take an endless number of forms, the discussion here is limited on finite mixture models. The most common form of which is likely the Guassian mixture model (GMM), used for a variety of applications including clustering, an unsupervised classification method, and traditional modelling. Mixture models are often used to cluster like data-points into groups. In the Bayesian paradigm this is done by inferring a latent classification variable for each data-point.

Bayesian Hypothesis Testing

Mon, 01 Jan 0001 00:00:00 +0000

The t-test is one of the most common tests used in statistics. It compares the means of two normally distributed datasets by calculating a test statistic and comparing it to the t-distribution. Variations of the test exist, including constant and differing variance between groups, paired and unpaired, and non-normality in the data. Here, Bayesian hypothesis testing is discussed, giving an indication of how a two-sample t-test would be conducted within a Bayesian context. Extensions to other situations are available, but the focus of this is to illustrate the how hyptothesis testing is done within the Bayesian paraidgm.

Introduction to Bayesian Hierarchical Models

Mon, 01 Jan 0001 00:00:00 +0000

Hierarchical models are synonymous with Bayesian methods, which is better equiped to be able to infer the parameters of large and complex models when compared to frequentist methods. They are popular as they allow simple building blocks to be combined to form a large and complex model. The ability of hierarchical models to reflect complex systems means they have been applied to a variety of modelling situations, within all aspects of science.

Introduction to Bayesian Statistics

Mon, 01 Jan 0001 00:00:00 +0000

Unlike frequentist statisticians, Bayesian statisticians believe that there is no ’true’ value of the parameters in a statistical model. Instead, Bayesian methods summarise parameter beliefs after observing data (a posteriori) by a probability distribution, giving more weight to more likely values. Parameter beliefs before observing any data (a priori) are called prior beliefs, and are similarly summarised by a probability distribution. Prior beliefs can be as vague or informed as required to reflect the beliefs of relevant experts. A lack of prior can be reflected by vague prior beliefs, with a high variance. In contrast, a large amount of prior knowledge from previous work may result in well-informed priors with high precision. Parameter beliefs are updated by combining prior beliefs and new evidence presented from a dataset. Bayesian methodology is fully probabilistic and considers model parameters, hidden states, as well as missing and observed data in the same vein. The coherent treatment of model parameters, data, and hidden states has allowed Bayesian methods to be used in a wide range of inference problems.

Introduction to Markov Chain Monte Carlo

Mon, 01 Jan 0001 00:00:00 +0000

In the Bayesian paradigm, it is often the case that a posterior distirbution cannot be found analytically, such as wehn analysis is conjugate. However, this is need not be the end. Markov chain Monte Carlo (MCMC) is a common method used to sample from a posterior distribution when the analysis is not conjugate. The premise of the method is to construct a Markov chain whose stationary distribution is the posterior density. Once the Markov chain has converged to the posterior, any sample generated by the chain will be a realisation from the posterior distribution and posterior beliefs can be inpsected by inspection of the posterior samples.