Markov Chains

In the Bayesian paradigm, it is often the case that a posterior distirbution cannot be found analytically, such as wehn analysis is conjugate. However, this is need not be the end. Markov chain Monte Carlo (MCMC) is a common method used to sample from a posterior distribution when the analysis is not conjugate. The premise of the method is to construct a Markov chain whose stationary distribution is the posterior density. Once the Markov chain has converged to the posterior, any sample generated by the chain will be a realisation from the posterior distribution and posterior beliefs can be inpsected by inspection of the posterior samples. ...

A Markov chain is a sequence of random variables whose current state depends only on the previous one and is independent of all states before it. This condition is known as the memoryless or Markov property. Their applications span many areas of statistics, including parameter inference and statistical modelling. The Markov property implies that the current state is a random variable whose possible values are described by a probability density dependent on the previous state. Being sequential, Markov chains are usually thought to progress with time, with updates occurring at discrete or continuous intervals. The state space of a Markov chain, the possible values of the random variables, can be either multi- or univariate and be discrete or continuous. Here we introduce both discrete and continuous state-space Markov chains with an example of both. The discussion is limited to discrete-time Markov chains, that iterate at fixed time intervals but continuous-time extensions exist. It is assumed that the reader is familiar with conscepts in statistics such random variables and probability density functions. ...