WHICH POLLS SHOULD YOU BELIEVE?

October 20, 2008 

Polling has come a long way since pollsters wrongly predicted Thomas Dewey would defeat Harry Truman in the 1948 presidential election. There were very few polls then, and they were only taken periodically. Pollsters were so confident that Dewey had the winning lead that they stopped taking polls days before the election and failed to capture the move toward Truman that took place in the final days of the campaign. Radio commentators and newspapers confidently predicted a Dewey win. The morning after the election, a smiling Truman posed for photographers holding a copy of the Chicago Tribune with the headline, “Dewey Defeats Truman.”

Today, there are so many pollsters and daily tracking polls that we are inundated by them. They've become an integral part of the election process; but have they become a more accurate predictor of electoral outcomes? Why is there often so much variation among different polls taken at the same time?

As Nick Timiraos writes in the Wall Street Journal, “Differences in predicting outcome result from how pollsters gauge voter turnout and weight party affiliation.” Pollsters routinely include a higher percentage of Democrats, anywhere from 3 to 12 percent more, in their samples. This, their experience tells them, better reflects the electorate and allows them to more accurately measure how people will vote.

The “Traditional” Gallup daily tracking poll, for example, is weighted to reflect ratios between Democrats and Republicans in previous presidential elections. The “Expanded” Gallup daily tracking poll assumes larger numbers of Democrats based on the larger number of people who affiliate themselves with the Democratic Party today. Last week in the Traditional poll, Obama had a 2-point lead. The Expanded Gallup daily tracking poll showed Obama with an 11-point lead. Which poll sampled the right mix of Democrats and Republicans?

This weighting technique has been troublesome for a long time. Anne Coulter, in her October 15, 2008, column, lays out the results of eight presidential elections from the Carter-Ford race in 1976 through the Bush-Kerry contest in 2004. Consistently, with the exception of the 2004 Bush-Kerry election, major polls conducted in October just prior to those elections reported a higher percentage of people who said they intended to vote for the Democratic candidate on Election Day than was the result.

Karl Rove suggests a similar phenomenon may be at work in many polls this year. Last week he wrote, “Barack Obama holds a 7.3% lead in the Real Clear Politics average of all polls, but the latest Gallup tracking poll reveals that there are nearly twice as many undecided voters this year than there were in the last presidential election. The Investor's Business Daily/TIPP poll (which was closest to the mark in predicting the 2004 outcome -- 0.4% off the actual result) now says this is a three-point race.”

And the weighting problem doesn’t take into account the Bradley effect, which, up until now, has not been a factor in US presidential elections. Tom Bradley, the African-American Democratic candidate for Governor of California lost the 1982 election to Republican George Deukmejian. Polls gave Bradley a significant lead going into the election, but he narrowly lost the race. Researchers determined that about five to six percent of people polled said that they were undecided or likely to vote for Bradley, but they voted for Deukmejian. We saw a similar trend in this year's Democratic primaries. Barack Obama’s standing in the polls frequently gave him an advantage over Hillary Clinton that was not reflected in election results.

Some believe, however, that the tendency to over-weight for Democrats and the Bradley effect will be countered in this year's national election by the large number of new young voters, if they turn out, and the overwhelming number of African Americans who will vote for Obama. Some even believe we'll see a Bradley effect in reverse, where people plan to vote for Obama but won't admit it to a pollster.

Add to this confusing mix the fact that not since 1944 has another issue so competed with the election for domination in the news. Then it was World War II. Today it’s the global financial crisis. Nearly everyone agrees that the state of the economy works against Republicans, possibly tipping the scales even more in Obama’s favor.

How these phenomena will affect the result of the 2008 election, if at all, we won’t know until it’s over. The Obama campaign, flush with money, is outspending the McCain campaign, limited to public financing, by a wide margin. Obama, exuding a sense of confidence that his election is inevitable, is campaigning in "red" states won by George Bush in 2004. Yesterday, Colin Powell endorsed Obama. If you're a gambler, the odds say Barack Obama will be our next president. Personally, I wouldn't bet against John McCain. All we know for certain at this point is, regardless who wins, pollsters, politicians, political scientists, and historians will be studying the 2008 election for a long time to come.