J. B. Miller, A. Sanjurjo: Surprised by the Gambler’s and Hot Hand Fallacies? A Truth in the Law of Small Numbers. 2016. IGIER Working Paper #552. http://dx.doi.org/10.2139/ssrn.2627354

Abstract:

We find a subtle but substantial bias in a standard measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data. The mechanism is a form of selection bias, which leads the empirical probability (i.e. relative frequency) to underestimate the true probability of a given outcome, when conditioning on prior outcomes of the same kind. The biased measure has been used prominently in the literature that investigates incorrect beliefs in sequential decision making — most notably the Gambler’s Fallacy and the Hot Hand Fallacy. Upon correcting for the bias, the conclusions of some prominent studies in the literature are reversed. The bias also provides a structural explanation of why the belief in the law of small numbers persists, as repeated experience with finite sequences can only reinforce these beliefs, on average.

Found via Andrew Gelman’s blog, which has a nice summary of the paper.

tl;dr. Probability is hard. Especially so, if you mix in conditional probability and finite sequences.