probability theory



Probability theory is concerned with mathematical models of phenomena that exhibit randomness , or more generally phenomena about which one has incomplete information.

Its central mathematical model is based mostly on measure theory. So from a pure mathematical viewpoint probability theory today could be characterized as the study of measurable spaces with a finite volume normalized to 11.

Broader perspectives may stress the relevance of other pure mathematical concepts for probability theory, or include aspects of the interpretation of mathematical results to phenomenology, the latter part making naturally contact with the field of statistics.

Notice that in this respect probability theory has a similar status as (other(?!)) theories of physics: there is a mathematical model (measure theory here as the model for probability theory, or for instance symplectic geometry as a model for classical mechanics) which can be studied all in itself, and then there is in addition a more or less concrete idea of how from that model one may deduce statements about the observable world (the average outcome of a dice role using probability theory, or the observability of the next solar eclipse using Hamiltonian mechanics). The step from the mathematical model to its use as a tool for making statements about the observable world is subtle, maybe a subject of philosophy, but in any case outside of the realm of mathematics. In probability theory the meaning of this step is traditionally a cause of debate, with two antagonistic main schools of thought being the frequentist interpretation and the Bayesian perspective on the nature of the relation of probability theory to the observable world.


Basic theory

Random variables are defined typically in terms of probability spaces, cf. the basic entries on measure space, probability space, conditional probability. Modern point of view is emphasising that many facts on random variables do not depend much on the choice of the probability spaces; the random variables are also often identified with their distributions.

Some argue that in the study of measure and probability, one should start not only with sigma algebra of measurable sets but also another of null sets. Somehow this is abstractly captured by the approach of commutative von Neumann algebras.

Some argue that th

Stochastic processes


ergodic process


Statistical Manifolds

Families of probability distributions often form statistical models, that is, submanifolds of the space of all probability measures on a sample space. Techniques from differential geometry may be applied in a theory known as information geometry.

Probability theory from the nPOV

We describe here some perspectives on (parts of) probability theory from the categorical point of view (see nPOV). This perspective mainly applies to the study of situations involving Markov kernels and Chapman-Kolmogorov property.

Prakash Panangaden in Probabilistic Relations defines the category SRelSRel (stochastic relations) to have as objects sets equipped with a σ\sigma-field. Morphisms are conditional probability densities or stochastic kernels. So, a morphism from (X,Σ X)( X, \Sigma_X) to (Y,Σ Y)( Y, \Sigma_Y) is a function h:X×Σ Y[0,1]h: X \times \Sigma_Y \to [0, 1] such that

  1. BΣ Y.λxX.h(x,B)\forall B \in \Sigma_Y . \lambda x \in X . h(x, B) is a bounded measurable function,
  2. xX.λBΣ Y.h(x,B)\forall x \in X . \lambda B \in \Sigma_Y . h(x, B) is a subprobability measure on Σ Y\Sigma_Y.

If kk is a morphism from YY to ZZ, then khk \cdot h from XX to ZZ is defined as (kh)(x,C)= Yk(y,C)h(x,dy)(k \cdot h)(x, C) = \int_Y k(y, C)h(x, d y).

This is based on earlier work by Michele Giry, see Giry's monad.

  • Michèle Giry, A categorical approach to probability theory Categorical aspects of topology and analysis (Ottawa, Ont., 1980), pp. 68–85, Lecture Notes in Math., 915, Springer.

Panangaden’s definition differs from Giry’s in the second clause where subprobability measures, rather than ordinary probability measures, are allowed.

Panangaden emphasises that the mechanism is similar to the way that the category of relations can be constructed from the power set functor. Just as the category of relations is the Kleisli category of the powerset functor over the category of sets Set, SRelSRel is the Kleisli category of the functor over the category of measurable spaces and measurable functions which sends a measurable space, XX, to the measurable space of subprobability measures on XX. This functor gives rise to a monad.

What is gained by the move from probability measures to subprobability measures? One motivation seems to be to model probabilistic processes from XX to a coproduct X+YX + Y. This you can iterate to form a process which looks to see where in YY you eventually end up. This relates to SRelSRel being traced.

There is a monad on MeasureSpacesMeasureSpaces, 1+:MeasMeas1 + -: Meas \to Meas. A probability measure on 1+X1 + X is a subprobability measure on XX. Panangaden’s monad is a composite of Giry’s and 1+1 + -.

The opposite of the Kleisli category of Giry's monad has as morphisms XYX \to Y, linear maps from bounded functions on XX to bounded functions on YY, which send the characteristic function on XX to the characteristic function on YY.

For more details on Giry’s monad and its variants see Giry's monad.


  • Quantum mechanics studies complex probability amplitudes whose absolute square can be interpreted as usual probability in the process of measurement, .e. quantum reduction. An alternative approach via Wigner's function? has real, but possibly negative probability.

  • noncommutative von Nuemann algebra?s can be interpreted as a noncommutative measure theory, see Alain Connes’ book Noncommutative Geometry.

  • free probability? theory of Voiculescu and others is another noncommutative generalization, with physical applications related to random matrix theory



Related concepts: expectation value, Radon-Nikodym derivative, cumulant, ergodic theory, statistics, stochastic process?, Wiener integral?

For references related to Giry's monad and variants see there.

For big picture in probability theory see answers to

An instance of a “categorical thinking” (in a generalized sense) in solving probability problems is a solution to Buffon’s noodle problem (wikipedia) discussed by Tom Leinster at nCafe here.

just as the natural numbers can be defined abstractly without reference to any numeral system (e.g. by the Peano axioms), core concepts of probability theory, such as random variables, can also be defined abstractly, without explicit mention of a measure space; we will return to this point when we discuss free probability later in this course.

  • John C. Baez, Jacob D. Biamonte, A course on quantum techniques for stochastic mechanics, pdf

Revised on March 20, 2014 11:40:03 by Tobias Fritz (