77: Are You Smarter Than A Mouse

    Before we begin, I'd like to thank listener Vinny from Virginia,
who recently posted a nice review of this podcast on iTunes.  Now,
let's get started.
    Suppose you try this experiment:  you create a small maze with two
possible paths, only one of which leads to some cheese.  You roll some
dice to create a 60% probability that the cheese will be on the left
path, and a 40% probability that the cheese will be on the right.  You
run a thousand or so trials with a single mouse, and then watch what
he does for the next hundred trials.  How well do you think he would
do?  As it turns out, in this experiment, the mouse quickly figures
out that the left path has cheese a lot more often than the right, and
after a sufficient number of trials has occurred, he will always head
left.  This means that on average he will get cheese in 60 of the 100
new trials, a pretty good result.
    Now print out the history of the mouse's thousand initial trials,
and show it to a human subject, without telling them your formula for
deciding where to place the cheese.  Ask the human to recommend what
the mouse should do for the next hundred trials.  Experiments have
shown that the human will recommend some pattern that involves both
the left and right paths at varying times, and only score about 52%
here-- a significantly worse result than the mouse gets!  What's going
on?
    The problem is that we have a natural human tendency to look for
patterns.  It's virtually a psychological certainty that if you stare
at a large number of random values, you will start to hypothesize
patterns there.  Suppose there is a run of seven right-cheeses
starting at trial 500-- maybe this means that every 500th iteration,
there will be seven right-cheeses in a row.  Or maybe you see that of
the 15 perfect even squares below 1000, all had a left-cheese, so that
must mean left cheeses are associated with perfect sequares.  While the
chance of any particular pattern like this are tiny, you can be sure
that after the fact, you'll be able to fit some pattern to the random
results.  And thus you'll miss the forest, the overall frequency and
the general randomness, for the arbitrary trees that pop up.
    In an experiment with rats, this is just amusing.  But when you
pick up a newspaper and read about real-life predictions, you should
always keep this experiment in mind.  Is some stock guru really
describing a useful investment strategy, or is he just back-fitting
some weird theory on random luck?  Was some terrorist attack due to a
foreign policy decision taken last week, or was it just a coincidence?
Will eating shark-cartilage-wrapped raw brussels sprouts floating in a
mixture of red wine and castor oil at every meal protect you from
cancer of the big toe, or were these patterns just back-fit onto a
random study?  Not that any of these theories are necessarily wrong--
but you need to think about the effects of true randomness in order to
make an informed decision. 
    And this has been your math mutation for today.

New Yorker article on predicting the future

Fooled By Randomness, a cool website on misunderstanding randomness and related topics