In a previous post, we briefly referenced an oft-cited polling mistake in the 1936 election. In honor of election season in the United States, we wanted to revisit the story and tell it in more detail.

The Story

It was 1936 and the Great Depression was 7 years old. The incumbent United States President, Franklin D. Roosevelt, had taken bold steps including his “New Deal.” The “New Deal” included many programs designed to assist Americans struggling under the depression, arguably at the expense of those who were doing better financially.

Roosevelt was up for re-election in 1936 and faced Republican Alf Landon.

Meanwhile, the Literary Digest, an influential weekly magazine of the time, had begun political polling and had correctly predicted the outcome of the previous five presidential elections. For this cycle, they had polled a sample of over 2 million people based upon telephone and car registrations. The results they obtained predicted Landon would win in a landslide with over 57% of the popular vote.

landon

However, there was a problem with the sample frame. During the Depression, not everyone could afford a car or a telephone. Those who did were usually wealthier, and therefore less likely to be directly helped by “New Deal” programs. As a result, this group was more likely to disapprove of Roosevelt than the general population.

So how far off were they? Check out these numbers…

The Prediction: Landon in a Landslide

Landon, 57.1%, Roosevelt, 42.9%

This projected Landon would win 370 out of 539 possible electoral votes.

Instead, the actual results gave a very different picture.

Actual Result: Roosevelt Runs Away With It

Roosevelt 60.8%, Landon 36.5%

Roosevelt won all but two states and 8 electoral votes en route to won of the largest landslides in presidential election history.

Why We Talk About It

What do we learn from this? An incorrect sample frame can destroy a study, regardless of the sample size. The researchers surveyed over 2 million people (today’s typical political survey asks between 500 & 1000 respondents), yet it missed about as badly as possible.

Also, sample size isn’t everything. Once you reach a certain number of respondents (typically around 500) additional responses begin to deliver diminishing returns.