The 1936 Election – A Polling Catastrophe
In a previous post, we briefly referenced an oft-cited polling mistake in the 1936 election. In honor of election season in the United States, we wanted to revisit the story and tell it in more detail.
It was 1936 and the Great Depression was 7 years old. The incumbent United States President, Franklin D. Roosevelt, had taken bold steps including his “New Deal.” The “New Deal” included many programs designed to assist Americans struggling under the depression, arguably at the expense of those who were doing better financially.
Roosevelt was up for re-election in 1936 and faced Republican Alf Landon.
Meanwhile, the Literary Digest, an influential weekly magazine of the time, had begun political polling and had correctly predicted the outcome of the previous five presidential elections. For this cycle, they had polled a sample of over 2 million people based upon telephone and car registrations. The results they obtained predicted Landon would win in a landslide with over 57% of the popular vote.
However, there was a problem with the sample frame. During the Depression, not everyone could afford a car or a telephone. Those who did were usually wealthier, and therefore less likely to be directly helped by “New Deal” programs. As a result, this group was more likely to disapprove of Roosevelt than the general population.
So how far off were they? Check out these numbers…
The Prediction: Landon in a Landslide
Landon, 57.1%, Roosevelt, 42.9%
This projected Landon would win 370 out of 539 possible electoral votes.
Instead, the actual results gave a very different picture.
Actual Result: Roosevelt Runs Away With It
Roosevelt 60.8%, Landon 36.5%
Roosevelt won all but two states and 8 electoral votes en route to one of the largest landslides in presidential election history.
Why We Talk About It
What do we learn from this? An incorrect sample frame can destroy a study, regardless of the sample size. The researchers surveyed over 2 million people (today’s typical political survey asks between 500 & 1000 respondents), yet it missed about as badly as possible.
Also, sample size isn’t everything. Once you reach a certain number of respondents (typically around 500) additional responses begin to deliver diminishing returns.
March 26, 2020
Social distancing in market research: Pivoting in-person methodologies
March 20, 2020
4 things market researchers need to think about in the time of COVID-19
February 26, 2020
Calling all brand admins: Join the Qualtrics admin training at XM Basecamp Live
February 14, 2020
Major League Baseball’s Scott Harris on surprising insights and the future of research
February 13, 2020