The scope of probability

Probability is the formalization of the study of the notion of uncertainty. The effects of blind chance are apparent everywhere. Biologically, we are all a random mixture of the genes of our parents. Catastrophes, like oil spills, volcano eruptions, tsunamis, or earthquakes, and happier events such as winning lottery prizes, randomly and dramatically change peoples’ lives.

Many people have a good intuitive understanding of probability. But this understanding can go astray when you have an initial idea about the likelihood of something, but then some new fact, whose relevance is not wholly apparent, is revealed. There are indeed a few notorious ‘trick questions’, about birthdays, or families with two children, or television game shows with three choices, that seem to have been designed to persuade you that the subject defies common sense. It does not. So long as any hidden assumptions in these questions are flushed out, and taken account of, sensible answers emerge. But probability does require clear thought processes.

The development of its ideas and methods has been driven by its wide applicability. The D-Day invasion of Normandy went ahead in June 1944 only because the probability of favourable weather was deemed sufficiently high. Engineers in the Netherlands must take account of the chances of severe floods when they build the dykes that protect their country from the sea. Is a new medical treatment more likely than present methods to enable a patient to survive for five years? How much you pay to insure your life, car, house, or possessions depends on the chances of an early claim being made. Most decisions you make – what to study at school, who to select as a life partner, where to live, which career to follow – are made under conditions of uncertainty. As Pierre-Simon Laplace wrote in 1814:

…the most important questions in life are, for the most part, only problems in probability.

Whenever the phrase ‘the probability is…’ appears, some assumptions (that may inadvertently have been omitted) are being made. If those assumptions are unwarranted, little reliance should be placed on the claim. I hope that, in this book, these assumptions are clear, either implicitly or explicitly. Before we look at how probability statements can be interpreted, we will describe different ways in which they may arise.

The objective view

The　classical , or　objective , view of probability is that often used during games of chance, such as rolling dice, or spinning roulette wheels. There is some list of outcomes: then, either from considerations of symmetry, or because we can find no good reason for one of them to occur rather than another, we take them all as equally likely. So we just count the number of outcomes, and give them all the same probability. Then the probability of any event in the experiment is taken as the　proportion of outcomes that favour it.

For example, when a coin is thrown twice, the four possible Head/Tail outcomes are HH, HT, TH, TT. With a　fair coin, H or T will be equally likely each time, so none of those four outcomes should be more or less likely than any of the others, each should have probability 1/4. Three of them contain Heads at least once, so the probability of the event that Heads appears at all is 3/4.

There are 1,326 ways of dealing a hand of two cards. (Take my word for it.) If the deck has been well shuffled, we take all these hands as equally likely. And 64 of them consist of an Ace and a ‘ten-card’ (i.e. Ten, Jack, Queen, or King), so we conclude that the probability of being dealt such a hand – ‘Blackjack’ – is 64/1326, just under 5%.

So far as probability considerations are concerned, both these examples could be reformulated in terms of choosing one ball from a bag of identical balls. The first bag would have four balls, three of them Red, the second 1,326 balls, 64 of them Red. Indeed, every example in this objective approach to probability is essentially identical to some problem about selecting one ball from some bag or urn (which perhaps explains the plethora of such exercises in student textbooks).

I emphasize that it is not enough to count the number of possible outcomes, and how many of them favour the event in question. There must also be no cogent reason for any outcome to be more or less likely than some other. Otherwise, you could fall into the trap of believing that your chance of winning the Jackpot in a Lottery is 50%, on the grounds that there are just two alternatives, either you win or you do not!

Experimental evidence – frequencies

We hope that dice used in household games like Monopoly, or casino games like Craps, will show each of their six faces equally often. But if a die is made from non-uniform material, or its width, breadth, and height differ, it is not sensible to assume that all outcomes are equally likely. Over a series of throws made under the same conditions, the frequency of any face will fluctuate, but will eventually settle down close to some particular value.You do not find that 20% of the first thousand throws are Sixes, and then the proportion among the next thousand throws leaps up to 60%. In these repeatable experiments, the outcomes may not be equally likely, but each of them has a propensity to occur at some characteristic frequency, and a　frequentist takes this value as the probability of that outcome.

Perhaps we get 170 Sixes in the first thousand throws of an imperfect die, then 181 Sixes in the next thousand, and so on. We can never deduce an　exact value for the probability of a Six from these experiments, but the data lead to estimates, and the more data that are collected, the better we expect the estimate to be. The fact that we cannot know the exact probability does not deny its existence. If I draw one card from a well-shuffled pack, there seems no reason for one suit to be favoured over any other. Each suit would have objective probability of 1/4. And if I return the card, reshuffle, and perform this task one hundred times, I expect each suit to arise about equally often, in this case about twenty-five times. Similarly, with ordinary dice where all six outcomes are intended to be equally likely, the chance of a Five on any throw is objectively taken as one-sixth: and over six hundred throws, we expect a Five on about one hundred occasions.

When experiments with equally likely outcomes are repeated often, the relative　frequency of any particular outcome is expected to be a close match to its probability, as calculated objectively. A fair coin seldom gives exactly fifty Heads in one hundred throws, but intuition does not tell you how close to that ideal you should reasonably expect. Frequency ideas are applied more widely than to repetitions of the same experiment under identical conditions. Will some imminent birth be male or female? With no specific information about the family in question, turn to data gathered from many countries and cultures over a long period. There is a consistent pattern that, for every 49 female births, there are 51 males. On the basis that there is nothing to pick out this birth from all others that are taking place, a frequentist will put the probability of a boy at 51%.

Some experiments on a heroic scale have been conducted. In 1894, the zoologist Raphael Weldon reported the results of more than twenty-six thousand throws of a set of a dozen dice. His data were not consistent with the idea that all six faces were equally likely, as the numbers five and six occurred rather too often. His dice had small holes drilled in each face to identify its score, and the faces for five and six are opposite two and one respectively. The centres of gravity of these dice will be closer to the faces with small numbers, giving a plausible explanation for the observed excess.

About seventy years later, Willard Longcor, a meticulous man with time on his hands, offered his services to top Harvard statistician Frederick Mosteller. Under Mosteller’s guidance, Longcor collected over two hundred dice, and threw each of them twenty thousand times, recording the outcome simply as even or odd – over four million data values. To make the conditions as near as possible identical, he used a carpeted desk-top, with a raised step to bounce the dice off. For cheap dice like those used by Weldon, there was a small but distinct bias towards too many even numbers – again, not totally unexpected because of the drilling. However, with the high quality precision dice as used in Las Vegas casinos, where the pips are either lightly painted or are extremely thin discs, no such bias was found. Frequencies with those dice were consistent with the classical view of equally likely outcomes.

Blackjack expert Peter Griffin noted wryly that, for a sequence of 1,820 hands he played in Las Vegas, the dealer’s upcard was either a Ten-card or an Ace on 770 occasions. The objective chance of receiving one of those favourable cards is 5/13, so Griffin wondered whether or not he had been cheated – random chance would give the dealer these good cards only 700 times on average.

In 2002/3, 6,202 children under five years old were admitted with suspected pneumonia to hospitals in Malawi, and 523 died, a fatality rate of 8.4%. Provided that there were no special circumstances making this period atypical, a frequentist would conclude that the probability of death when a young Malawi child catches pneumonia is about 8–9%. From an objective perspective, making general statements about the chance of death among young Malawi children with pneumonia would be speculation, albeit based on evidence: but all that can be said for certain is that if one of　those particular 6,202 children were selected at random, the chance that child died was 8.4%.

The relationship between frequency data and objective probabilities will be further explored later.

The subjective interpretation

Bruno de Finetti, one of the most in fluential thinkers in the field, wrote

PROBABILITY DOES NOT EXIST

As Professor of the Theory of Probability, he was not dismissing his subject as a mirage, rather he rejected　absolute claims such as ‘The probability of Heads is one half’. To him, every statement involving a probability is just an expression of opinion, based on one’s own experience and knowledge, and perhaps changing when more information arrives.

Consider the five assertions:

The England cricket captain will win the toss in England’s next Test Match;

Whoever wins the Oscar for best actor next year will also win it the year after;

No person born in Oslo has yet won an Olympic fencing gold medal;

Richard III was responsible for the death of the Princes in the Tower;

Al Gore would have been elected US President in 2000 if Ralph Nader had not stood as a candidate.

To each of them, we can offer our　degree of belief , or personal probability , or　subjective probability . This will be some nonnegative number, not greater than 1: equivalently, it is a percentage between 0% and 100%, inclusive.

Zero and one represent, respectively, the two extremes of impossible , and　certain . I am certain that the soccer World Cup will be hosted by an African nation again during the present century. I think it is impossible for someone under twenty years of age to win a Nobel Prize.

Assessing subjective probabilities

The five assertions above have different natures, and we have different kinds of evidence about them. For the first, we might appeal to symmetry between Heads and Tails. For the second, we have the history of the Oscars since 1929 to guide us. In both these cases, the truth or otherwise of the statement will become known within a finite time. The third is either true or false, and could be established now by a thorough trawl of Olympic records. The fourth is also either true or false, but we will never know which. We cannot rerun history to ascertain the truth or otherwise of the fifth claim.

Specific examples later will illustrate how subjective probabilities have been assessed. Aside from those arguments, there are at least three distinct general approaches. One is as the　fair　price for a bet that the event will occur. But this does not work for everybody: some people have principled objections to betting,others are unwilling to contemplate actions that might ever lead to a loss. And even for those who do feel comfortable with betting, their fair price might differ according as to which side of the bet they were on.

A second way to assess your degree of belief in an event uses the objective approach. Which offer would you prefer: to receive ￡5 if the event occurs, or to receive ￡5 if you correctly guess the colour, Red or Black, of the top card in a well-shuffled deck? If you prefer the latter, your degree of belief in the event is below 50%.

Suppose that is the case. Now compare the prospect of receiving ￡5 if the event occurs, or getting it if you correctly guess the　suit of a randomly drawn card. The latter should occur 25% of the time, so your preference here will tell you whether your degree of belief is below 25%, or is between 25% and 50%.

More elaborate comparisons along these lines let you home in on a situation where you cannot say on which side your preference lies. Your degree of belief in the event will then be close to the objective probability of the corresponding card selection. Rather than use a deck of 52 cards, with its awkward fractions, you might think of an urn containing 20, or maybe 100, identical balls with which to specify the alternative events.

Give your answers with appropriate precision. Tennis players John Isner and Nicolas Mahut played the longest match in Wimbledon history in 2010; via counting, the chance that they would be drawn together again the next year (it happened!) is precisely 2 in 285, perhaps better rounded to ‘a little under 1%’. But it was absurd of　Star Trek ’s Mr Spock to tell Kirk that the odds against their escape in one episode were ‘approximately 7,824.7 to 1’.

For a third method, think of a modest sum of money, not so small that you are totally indifferent to it (say, one penny), nor so large that possessing it would make a dramatic difference to your circumstances (￡1 million to most people, rather bigger for Bill Gates). For me, ￡10 fits the bill – call this　unit amount . Now suppose that, somehow or other, the truth or falsity of the event will be revealed tomorrow: and you will receive this unit amount if it is true, but zero if it is false. But rather than wait for tomorrow, you could receive a definite proportion　p of this unit amount today. (Getting the money today or tomorrow makes no difference to you.)

If　p is tiny, you are likely to reject the offer, and will prefer to wait; if it is close to unity, you may well accept that definite amount. But there will be some intermediate value of　p where you are indifferent between taking this offer, and waiting for the outcome to be revealed. This　p is your degree of belief about this statement or event in question.

I offer my own subjective answers for the five assertions above. I can think of no sensible reason why one side should be more or less likely to win a cricket toss than the other, so my first figure is 50%. Looking at Oscar history, not only for actors but also the other categories, the award has occasionally been repeated in successive years: perhaps there are more candidates these days, leading me to suggest 3%, or lower. Norwegians are not noted for fencing, but we have épée, foil, sabre, and the sport has appeared in all the Summer Games from 1896. Some native of Oslo might have won sometime, but I strongly doubt it – my figure here is about 95%. Prejudice in favour of the White Rose county, rather than objective evidence, leads me to suggest 10% for the fourth claim. For the fifth claim, considering the votes in each State, and thinking of a plausible division of the votes Nader received, guide me towards 20%.

Pause a while, and make your own suggestions for these five claims. The better you are at assessing probabilities when matters are uncertain, the more likely you are to be happy with the decisions you make in life.

Odds

Whether we use the classical approach, or frequencies, or degrees of belief, the term　odds is often used when describing probabilities.

We might say that the odds of obtaining a Six with a fair die are ‘five to one against’ – for every time we get a Six in a sequence of throws, we expect not to do so five times. If an outcome is expected to be more likely than not, such as the higher ranked player winning a tennis match, that event is said to be　odds on .

There is an exact correspondence between probabilities and odds, and we can switch easily between them. Thinking of frequencies can help. If the probability is 20%, or one fifth, we expect the event to occur in one occasion out of five, so the odds are ‘four to one against’. For a probability of 75%, we expect it to occur three times out of four, giving odds of ‘three to one on’. And if the odds are stated as six to five against, this indicates that for each five times the event happens, it fails to do so six times, so its probability is 5/11.

You do not have to stick to whole numbers. The probability that the top card in a well-shuffled deck is either a King or a Queen will be taken as 2/13. This could be quoted as ‘eleven to two against’ or, equally accurately, ‘Five point five to one against’. Use whichever you like.

Although the phrase ‘the odds are one to one’ is never used, it would make perfect sense. It indicates that an event is expected to happen just as often as not, so its probability is one half. Instead, with a straight face, we say ‘the odds are evens’.

Issues to resolve

There are no important disagreements about how to work with probabilities, but adherents of the three approaches we have described may deduce their values in different ways. Each perspective has its uses. In seeking to understand how the subject works, we will appeal to whichever viewpoint appears appropriate.

The objective approach is limited to circumstances having finitely many outcomes, all judged equally likely. But no coin or die is perfectly symmetrical, and on what basis can we dismiss its imperfections as irrelevant? Can we even be sure that we agree on the number of possible outcomes? For example, suppose we are told that an urn contains two balls, either both White, both Black, or one of each colour. Should we argue we have　three equally likely cases, or that there are really　four equally likely cases, arising when the balls were inserted, in order, either as WW, WB, BW or BB? These different outlooks would give different answers for the chance that both balls are Black. Or suppose you reach a road junction with three possible exits, two of them leading to Newtown, the third to Seaport: making a ‘random choice’, is the chance that you aim for Seaport one third (one exit in three), or one half (one of two destinations)?

A frequentist seeks to deal with circumstances that are repeatable indefinitely often under identical conditions. The number of outcomes need not be finite – think of tossing the same coin until Heads appear three times in succession, or selecting a random point on a stick. But, however much care we take, the experimental conditions cannot be　absolutely identical, and any limiting value can only be estimated. How should the error in this estimate be described? Claiming that the probability is at least 99% that the error is under 2% requires a circular argument – we need to know what probability is, in order to define it!

For questions such as the probability that one country invades another, or the chance that a particular heart transplant is successful, the circumstances arise once only, and the alternatives cannot be reduced to a finite list of equally likely cases. The objective and frequency approaches are silent on these matters. A subjective approach is required.

A subjectivist must ensure that her beliefs are consistent with each other. For example, in the UK National Lottery, a machine selects six numbers from the list {1, 2, 3,…,49}, and Susie may be content to take all 14 million or so possible selections as equally likely. Then, when asked which is more likely,

(a) that no number drawn exceeds 44, or

(b) that those drawn do not include two consecutive numbers,

she may, after a little thought, come down on one side or the other. But if she selects　either of these events as more likely than the other, she will be guilty of inconsistency, as proper counting shows that they can occur in exactly the same number of ways! Nothing in the subjective approach specifies how such an inconsistency should be resolved, merely that it must be.

Because we wish to think about probabilities in circumstances wider than when there are finitely many equally likely choices, and in circumstances that cannot be repeated indefinitely often, we will take the subjective approach as the default option. But we are likely to hold more firmly to our opinions when they are backed up by either an objective, or by a frequency, argument.

Interpretations

Using the ‘balls in a bag’ viewpoint, the probability of some event is taken as the proportion of Red balls in the bag. So a value of zero can occur only if there are no Red balls, in which case the event will never happen. Similarly, a probability of unity corresponds to every ball being Red, so here the event occurs every time. These values, zero and unity, are the only ones that can be conclusively proved　wrong by experimental evidence: if the event happens, its probability cannot be zero,if it fails to happen, its probability cannot be unity. And this is true for the frequency, or subjective approaches also. So suppose the probability has some intermediate value, say 3/4.

We first dispose of one finicky point. No matter how well a roulette wheel has been engineered, it is physically impossible that all the numbered slots have　exactly the same chance. What the casino requires is that the chances are close enough to the ideal that it is inconceivable that any number could be picked out as more or less likely than another. Similar remarks apply to dice, coins, or cards. So statements like ‘the probability is 3/4’ will mean that the probability is close enough to 3/4 for all practical purposes. Otherwise, a pedant might smugly tell you that he knows that the probability is not 3/4, without fear of contradiction.

In the context of repeatable experiments, what do we expect to follow from the claim: ‘The probability of a Red ball is 3/4’? Emphatically, we do not expect that if we conduct this experiment four times (replacing the ball drawn on each occasion), we shall draw a Red ball in precisely three of them. It is possible that four repetitions throw up no Reds at all, or even that Red happens every time. But over a long series of repetitions, we do expect the overall frequency of Red to be close to 3/4.

There are no black/white answers to what constitutes a long series of experiments, nor to how close to 3/4 is acceptable. If I obtained Red only 20 times in the first 40 repetitions, I would have very strong doubts about a claim that the probability was 3/4; but those doubts would be largely assuaged if the next 40 repetitions gave 28 Reds. Believing or disbelieving this claim can be a provisional position for quite some time. Assuming the experimental conditions do remain unchanged throughout, use all the data collected to reach a decision – short runs can mislead.

I will offer some guidelines, and justify them later. Take the case when we make one hundred repetitions, and the supposed probability is some middling value, near one half. Compute the difference between this figure and the actual frequency from data: if this difference exceeds 0.1, I would have some doubts about the claim, and if it exceeded 0.15, I would have strong doubts. With a thousand repetitions rather than a hundred, I expect closer agreement, so replace those numbers by 0.03 and 0.05. If the supposed probability is closer to zero or unity, say 10% or 90%, I would also require better agreement. It can be much easier to be convinced, on the basis of repeated experiments, that a particular probability is　not some alleged value.

What about a subjective assessment, such as that the probability of rain tomorrow is 60%? We cannot recreate today’s weather conditions hundreds of times, and check how often it rains. This ‘experiment’ can be conducted once only. But we might test the claim by looking at the process that led to it being made. Forecasters use models of weather patterns to reach their conclusions, and even if the figure on their computer screen is 31.067%, they sensibly offer round figures. You hear ‘The chance of rain is about 30%’. So now you can collect data for different days, and look at the empirical evidence – in how many of the 83 days last year when the chance of rain was put at 30% did it actually rain? So long as that proportion was reasonably close to 30%, your belief in the method is reinforced, so accepting the figure given for ‘tomorrow’ is a rational response.

Probability is the key to making decisions under conditions of uncertainty. If you honestly believe that the probability of a particular event or statement is unity, you should act as though it will definitely occur; and if your honest belief is that the probability is zero, act as though it cannot occur.

If you think the probability is some value between zero and one, act as though you expect it to occur that proportion of the time.For example, if your judgement is that the probability is 60%, imagine that you will face this situation a hundred times, in sixty of which (but you have no idea which sixty) this event will happen, and forty times it will not. Swallow hard, and decide on your action, taking into account this balance. Had you judged the probability to be 80%, so that you expect the event to happen rather more often, your action might well be different.

As Bishop Joseph Butler wrote in his 1736 Analogy of Religion , ‘To us, probability is the very guide to life’.