Will Sampling Error Determine our Next President?

sampling error used in pollingOkay, that might be a bit of exaggeration. But now that I have your attention, let’s think about this!

The Republican Party, finding themselves in the unprecedented position of wrangling 17 presidential hopefuls, determined to limit the first debate, sponsored by Fox News, to the top ten candidates. But how to decide who are the top ten?

As we all know, political polling picks up during election years, and there are many polls, using different sampling methods, different questions, and different weighting and analytic techniques to measure voter intent. So which one is right? Sometimes, when perusing the media coverage of this election, it seems like either all of the polls are right – or all of them are wrong.

One of the problems is question wording and timing. The election is 16 months away! Are people outside of Iowa and New Hampshire even thinking about the election? And if so, how likely is it that they will change their minds about for whom to vote? After all, your favorite candidate might not even make it to the election (especially with 17 candidates on the Republican side!) So how do you ask voters what they are going to do next November?

However, the real heart of the matter is sampling. In the July 29 New York Times, Kevin Quealy and Amanda Cox played with sampling error to illustrate how the debate lineup could have been different. As they explained, “Consider one recent national poll, a survey of 336 registered Republican voters, conducted by Monmouth University from July 9 to 12. In this poll, Carly Fiorina was preferred by 1 percent of respondents — or about three people, if everyone counted the same. But in polling, responses are weighted based on demographics, so the makeup of respondents resembles that of the entire population. Take, for example, a 2012 Republican primary poll conducted for The New York Times and CBS News. In this poll, some respondents, including a young high-earning man in the West and a Hispanic woman, counted as much as four average people. So it’s entirely possible that Ms. Fiorina’s 1 percent came from a single person if this person came from a demographic group that is hard for pollsters to reach.” (See also our recent Infosurv Insider blog about sampling error.)

There are many questions about the ability of polls to predict voters’ behavior, as well as about whether anyone pays any attention to polls at all. (Especially this far in advance of the actual election.) There is no doubt that participation in televised debates is highly desirable for the candidates, even if it is only to appear on the same stage as all the other candidates. Using polls to identify who will or will not appear on that highly coveted national stage is a process fraught with peril, as the recent televised debate demonstrated.

The discussion about the role of polling in elections will continue long after President #45 is elected, and probably even after President 46! However, As American Entertain Robert Orben said, “Do you ever get the feeling that the only reason we have elections is to find out if the polls were right?”

 

Leave a Reply

Your email address will not be published. Required fields are marked *