\[ \newcommand{\Exg}{\operatorname{\mathbb{E}}} \newcommand{\Ex}{\mathbb{E}} \newcommand{\Ind}{\mathbb{I}} \newcommand{\Var}{\operatorname{Var}} \newcommand{\Cov}{\operatorname{Cov}} \newcommand{\Corr}{\operatorname{Corr}} \newcommand{\ee}{\mathrm{e}} \]

20  Markov chains in continuous space

Summary:

• Markov chains also exist for continuous state space, where they are defined by the transition density \(p(x, y)\).
• Properties are similar to Markov chains in discrete space: a stationary distribution can be found by solving the discrete balance equations \(\pi(x) p(x,y) = \pi(y) p(y,x)\); and the ergodic theorem says that \(\sum{1}{n} \sum_{i=1}^n \phi(X_i) = \operatorname{\mathbb E} \phi(X)\) where \(X\) has PDF \(\pi\).

Read more: Voss, An Introduction to Statistical Computing, Subsection 2.3.1.