Ten Great Ideas about Chance

This book grew out of a course given by the authors, a statistician (Diaconis) and a philosopher (Skyrms) to a mixed audience of students. The idea is that the reader should be familiar with a basic course on probability theory. If that has been a while and it needs some refreshing, then an appendix summarizes the main definitions and formulas in a brief tutorial. The true subject of the course are the foundations and the philosophical aspects of probability and statistics and how they evolved in the course of time. These are summarised in the form of ten "great ideas" developed in as many chapters that have formed the subject as we know it today. They are introduced in an approximately chronological order, so that the book is also a history book that shows the genesis of the ideas from Cardano to the 21st century.

A mathematical introduction to probability starts with a probability space. That is a triple $(Ω,\mathcal{A},P)$ with $Ω$ a non-empty set (sample space), $\mathcal{A}$ a σ-algebra (event space) and $P$ a probability measure defined on $\mathcal{A}$. This is more or less how A.N. Kolmogorov has defined probability, but the idea of chance has evolved in several stages before this definition emerged.

The real origin of probability is commonly believed to be in the correspondence between Pascal and Fermat (1654). Chance was measured for the first time by counting and defining a frequency of an event occurring in many repetitions of the same experiment like throwing dice. Huygens and later Jacob Bernoulli wrote the first books on probability (1713) and even defined conditional probability. In fact, they also considered another problem where throwing the dice was part of a game where winning or loosing was the issue. The gambler had to estimate his chance of winning and adapt his stakes as the game went on. In that setting probability was expected to predict the future. What will be the future outcome of a dice throw and eventually of the game, or of the economical evolution, or of the political elections, if we stretch it to the present day. Your actions today will depend on your expectations. You buy on the financial market when expecting A and you sell when expecting B. It depends on your judgement and how much you value the outcome A by the money you hope to win if A occurs. That is judgemental probability and this can be considered a measure of probability on condition that it is coherent. If not, you get a Dutch book and you can be drained of all your money. Clearly this is a dynamical system, thus as new evidence becomes available, your judgement should be adapted. But people are not always rational or do not act accordingly. Their acts depend also on psychology. The Allais paradox shows that people value certainty more. Even if it is objectively preferable to choose A involving risk, rather than B which involves certainty, people will choose for certainty. They also prefer a choice formulated in terms of gains over an equivalent formulation in terms of losses. All this makes financial markets difficult to manage on a pure rational basis.

In Bernoulli's view, just counting the frequency of an event happening defines its probability, but what he did was basically the inverse. His law of large numbers is saying that for a given probability of an event, this will show up eventually as the frequency of that event when the experiment is repeated a large number of times. In fact, this is a bit of a shaky definition, not built on a sound ground. Sufficiently often repetition of identical experiments basically means an infinite number of times and that is practically impossible. It was John Venn who made such a statement mathematically more rigorous but still a vivid debate was started in Victorian England whether frequency and probability should be identified or whatever relation existed between both. Von Mises tried to answer Hilbert and attempted to give an axiomatic definition but the discrepancy between reality and ideal randomness remained.

The definition of Kolmogorov (1933) was a release. He defined a random variable, and probability was not a frequency but it could take any value in an interval $[0,1]$ and he used the newly developed notion of measure. But it still needed infinity and the controversy about the inference from frequency to chance continued. This controversy is related to inductive reasoning. Is what we observe in a large number of experiment guaranteed to be consistently observed if the number of experiments is increased to infinity? Because the sun has risen every morning till today, are we sure that will happen again tomorrow? Can we deduce behaviour from observation? Especially David Hume was sceptical about inductive reasoning. Bayes applied conditional probability to use the evidence that becomes available with every observation to reconsider the probability that it will happen the next time. So one starts from a prior hypothetical distribution, a subjective belief so to speak, and its probability is updated permanently. This a posteriori updated model can then express a degree of belief (probability) to what will be the next observation. Laplace in France came to a similar conclusion. Bruno di Finetti's theorem proves the existence of such an a priory distribution that perfectly predicts the outcome of an exchangeable sequence (i.e. independent of reordering).

With the inference problem more or less settled, it remains to find a computer algorithm to generate a truly random sequence. Per Martin-Löf gave an appropriate answer (1966) using the theory of computability developed in the 1930's by Church and Turing. Consider a set of countably computable binary sequences and select a nested sequence of subsets whose probability behaves like $2^{-n}$, and with probability 1, you will end up with a set of sequences that are perfectly random (i.e. that have probability 0).

Statistical mechanics and quantum theory placed probability at the center of physical phenomena. The second law of thermodynamics explained on a statistical basis required a notion of ergodicity: the time average equals the ensemble average like Boltzmann believed in his model of gas dynamics. A full proof that it is indeed ergodic is still missing today. Poincaré introduced chaos in such systems and the sensitivity on the initial conditions will eventually remove the prior probability on the initial conditions, which leads to a Boltzmann-Liouville uniform distribution. Quantum physics leaves us with many puzzling mysteries, even for the designers and the specialists, but still it is possible to place it in a framework of classical probability. The EPR (Einstein, Podolsky, Rosen) paper of 1935 was based on probability and so stated the nonlocality (the spooky action at a distance) that was confirmed in subsequent experiments. We also need a form of quantum ergodicity, but it can be shown that that follows from classical ergodicity.

In a last chapter, inductive reasoning is reconsidered revising what has been explained in the previous chapters with the visions of Hume, Kant, and Popper and the sceptics Bayes, and Laplace. But there are always assumptions. Should we be sceptic about these assumptions too. But then one can be sceptic about anything making absolute scepticism impossible.

In summary, in this book the authors focus on the philosophical aspects of probability and statistics. The ten great ideas are captured in the titles of the chapters: measurement, judgement, psychology, frequency, mathematics, inverse inference, unification, algorithmic randomness, physical chance, and induction. But these are of course just keywords and much more is covered in each chapter. Their account is partially historical because they describe the origin and the controversies that arose about some concepts. Besides technical graphics, pictures of some of the main historical contributors are used as illustrations but the historical component is not so thorough that we get biographies of the mathematicians involved (not even short ones). Besides the summary on probability at the end of the book, several of the chapters have appendices in which some topic is further elaborated. There are a few short footnotes but most of the notes (mostly mentioning a reference) are collected per chapter at the end of the book. An annotated list of references per chapter is added separately. Often the vision of some historical person is formulated in his own words (or their English translations) as easily recognizable quotes. The typesetting is done very carefully (although I could spot a few typos) and they are just long enough to catch the main idea without boring, as if they were written to cover one lecture each.

Although the text is discussing formulas that are used in everyday practical applications, the reader should be warned that it is mainly about their foundations and philosophical aspects. Thus a reader who is just a practitioner may have difficulties if not interested in the philosophy. Philosophers not very familiar with the underlying mathematics may have a hard time too. But if the reading is tough, the tough go on reading, and when you worked your way through it, it is a very enriching journey. Your vision will be broadened assimilating all these issues and solutions as well as open problems from the early history of probability, game theory, financial markets, politics, thermodynamics, quantum theory and much much more.

Adhemar Bultheel
Book details

The authors start from the early days of probability applied in gambling and work their way through ten great ideas that have shaped the current day scenery of probability and statistics. The discussion is fundamental and covers the different philosophical interpretations of concepts such as frequency versus mathematical probability, Bayesian probability, subjective probability, entropy, quantum physics, and inductive reasoning. For the reader who needs to refresh notions from elementary probability a short tutorial is provided.

Author:  Publisher: 
978-0-6911-7416-7 (hbk)
£ 22.95 (hbk)