You have two coins, one of which comes up heads one-third of the time (coin A) and one of which comes up heads one-half of the time (coin B).
You pick one of these coins uniformly at random. Then you flip it twelve times, and it comes up heads five times. Which of the two coins is it?
The first instinct is to say that coin A is expected to come up heads four times, and coin B is expected to come up heads six times… five is halfway between those, so we don’t know anything.
But the standard deviation of the number of heads from A is and the standard deviation of the number of heads from B is ; so in standardized terms, maybe B is more likely?
So let’s use maximum likelihood. The probability of getting 5 heads in 12 flips from coin A is
and the probability of getting 5 heads in 12 flips from coin B is
The likelihood of coin B is larger, so we choose B.
Alternatively, this is equivalent to using Bayes’ theorem with a uniform prior. Say . Then we can find the posterior probability of having picked coin B, given 5 heads:
and this is 0.5033877; the evidence from five coin flips is very weak. The probability of having picked coin B given heads in tosses can be computed similarly, and it’s
After some cancellation, this is
and in the case this is
We might ask: is there a general decision rule? Fix n; when is ? If is less than that number then guess coin A was used; if was greater than that number then guess that coin B was used. To get we must have ; rearranging this is . Taking logs we get and so , slightly less than . If the proportion of heads is greater than 0.41503575 then guess coin ; if it’s smaller than guess coin .
This was inspired by an exercise from Freedman, Pisani, and Purves, Statistics, fourth edition (Chapter 24, exercise B1), with the language regularized: a thumbtack is thrown in the air. It lands either point up or point down. Someone suggests that the point comes up half the time; someone else, one-third of the time. How could you decide which model is correct? The answer given in the text is to toss the thumbtack many times and see whether the percentage of point-up times is closer to one-half or one-third. This is in spirit correct, but it turns out that 5/12 is not the optimal splitting point.