Suppose a fair coin is tossed $1000$times. If the first $100$tosses all result in heads, what proportion of heads would you expect on the final$900$tosses? Comment on the statement “The strong law of large numbers swamps but does not compensate.”

A fair coin is tossed$1000$times. If the first $100$tosses all result in heads, what proportion of heads would you expect on the final $900$tosses.

Let's say that the random variable$X_i$is equal to one if and only if in$i$th toss there was Head and otherwise it is equal to zero. Now, we are given that $X_i=1$for$i=1,\dots ,100$. By the law of the large numbers, we have that

$\underset{n}{\lim }\frac{X_1+\cdots +X_{100}+X_{101}+\cdots +X_{1000}}{1000}=E\left(X_1\right)=\frac{1}{2}\text{a.c.}$

so we would expect that $500$out of $1000$time there were Heads. Since we are given that the first $100$times there were Heads, we could think that in the remaining $900$tosses we would expect $500-100=400$Heads. But, that thinking is wrong! The law of the large numbers gives us almost certainly (probabilistic) convergence, not a deterministic value of expectation. The truth is that sequences $X_1,\dots ,X_{100}$$and$$X_{101},\dots ,X_{1000}$are independent, so we have that

$E\left[\frac{X_{101}+\cdots +X_{1000}}{900}\mid X_1=1,\dots ,X_{100}=1\right]=E\left[\frac{X_{101}+\cdots +X_{1000}}{900}\right]=\frac{1}{2}$

so the right answer is that we would expect $450$Heads in the remaining$900$ tosses. The strong law of large numbers swamps, but does not compensate.