Hebert Research - Polls: A Critical Look - by Jim Hebert, CEO

Bellevue, Wa, March 8, 2016 (Newswire.com) - ​​Polls

A Critical Look

By

Jim Hebert

This presidential year has dominated the news, but polls have as well.  In fact, the polls have become the news. There are, historically, the largest number of polling organizations involving colleges, polling shops, and the large, well-esteemed firms such as Gallup and Pew.  Gallup, this year, made the decision not to conduct 2016 presidential polls.  This is noteworthy, given the tremendous success of predictions in the Roosevelt victory.  Although, the Obama/Romney mistake of Gallup was a time for Gallup to re-exam their role in the 2016 polls.  So, what are the questions that must be asked about candidate polls at the national and local level?

The Sampling Frame

The two-party system once was much more defined, so were demographics and lifestyle groups.  This is no longer the case. The modalities of voter’s profiles which were discredited have become continuous variables.  From a sampling frame, this is a challenge to make sure the samples of the population are representative. It also means that tests of statistically significant differences require much larger sample cells. The sampling frame of the United States requires much larger samples and weighing of the samples. The results are that polls, in order to be accurate, are becoming increasingly expensive.

Most candidate samples are based on lists of past voters, comprising either of Republicans or Democrats, but not the increasing larger proportion of Independents, who are much more defined by issues than party loyalty.  Hebert Research calls this the “party of the intellect,” who weighs the issues based on content and creditability.  Even more important is the moderate Democrats and Republicans when the final choice will cross over.

The sampling frame must include the basic question - who will vote.  Far too many samples based on voter registration means voter issues have far become the case. Likely voters are essential.  What has happened is the disconnected, disenfranchised, and discouraged voter. This could not even be clearer with elections of local government officials and judges. What happens then is that the political election process is based more on what is called “election by participation.”  This means that special interest groups, such as unions or ARD associations, get on the vote for their pre-selected and chosen candidate. 

Sampling Methods

Prior to cell phones and social media, 96% of all households had a telephone.  This is no longer the case, especially with younger populations, who can only be reached to conduct a survey through cell phones and the Internet.  The pollster problem is an increasing high non-response error and under-sampling of the likely voters.  This can be over 70% of the sample unrepresented.  

Robocalls, or the use of automated dialing based on third-party list using, often rewards those respondents who select into the sample, but may not represent the actual voter.  The theory of probability samples is not what provides the representativeness, either on a simple or stratified basis. Larger, inexpensive numbers of respondents can be produced, which may well not be misrepresentative, but the sample size is used to impress and validate.

Online methods of conducting polls have the same problems as robo-methods, with the further limitation to only those with a computer and a cell phone app and who are willing and able to use this method.  The other problem is authentication of the respondent and deterrence of compromising personal security.

Exit polls are often quoted, but this method is only among voters who just voted.  Online voting or undersampling at the polling location of interviewers may not represent those who have not yet voted. 

Statistical Error

The accuracy of polls must address far more than merely the plus or minus of the overall poll.  Each question and subsample has its own margin of error and a given confidence level related to who is conducting the poll and who was included in the sample cell.  What this means is that the overall margin of error may be +/- .05, but young millennial women by pattern could be several times this margin of error.  Another error problem can easily occur with the respondent not accurately reporting his or her response or the researcher making input errors.  Time of day or day of the week can easily reflect differences.  The instantaneousness of overnight polls does not account for this error possibility.  Finally, there is the error of omission, such as urban voters of selected neighborhoods.  Where it is that the voter lives explains much of the voter answers.

At best, polls are an approximation and difficult to use to predict voter results.  There are many solutions to improving the quality of polls.  Statistical controls to determine error, validation, and verification methods and knowing who conducted the poll and the objectives are critical.