The 1936 election – a polling catastrophe

There is an oft-cited polling mistake that took place in the 1936 election. Read on to find out how an incorrect sample frame can destroy a study and lead to plenty of confusion.

The story

It was 1936 and the Great Depression was 7 years old. The incumbent United States President, Franklin D. Roosevelt, had taken bold steps including his “New Deal.” The “New Deal” included many programs designed to assist Americans struggling under the depression, arguably at the expense of those who were doing better financially.

Roosevelt was up for re-election in 1936 and faced Republican Alf Landon.

Meanwhile, the Literary Digest, an influential weekly magazine of the time, had begun political polling and had correctly predicted the outcome of the previous five presidential elections. For this cycle, they had polled a sample of over 2 million people based upon telephone and car registrations. The results they obtained predicted Landon would win in a landslide with over 57% of the popular vote.

However, there was a problem with the sample frame. During the Depression, not everyone could afford a car or a telephone. Those who did were usually wealthier, and therefore less likely to be directly helped by “New Deal” programs. As a result, this group was more likely to disapprove of Roosevelt than the general population.

So how far off were they? Check out these numbers…

The Prediction: Landon in a Landslide

Landon, 57.1%, Roosevelt, 42.9%

This projected Landon would win 370 out of 539 possible electoral votes.

Instead, the actual results gave a very different picture.

Actual Result: Roosevelt Runs Away With It

Roosevelt 60.8%, Landon 36.5%

Roosevelt won all but two states and 8 electoral votes en route to one of the largest landslides in presidential election history.

Why we talk about it

What do we learn from this? An incorrect sample frame can destroy a study, regardless of the sample size. The researchers surveyed over 2 million people (today’s typical political survey asks between 500 & 1000 respondents), yet it missed about as badly as possible.

Also, sample size isn’t everything. Once you reach a certain number of respondents (typically around 500) additional responses begin to deliver diminishing returns.