Poll Vault

[Brainster]

Rather than start a new thread for every poll that comes down the pike, how about if we have one main polling thread? This will enable us to look at trends over time, discuss problems with polling, etc.

Here's a new CNN/Opinion Research Poll that shows Barack Obama with a five point edge over John McCain.

Obama: 50%
McCain: 45%
The poll was conducted June 26-29 of 906 registered voters. Error margin is plus or minus 3.5 percentage points. When Nader and Barr were included, Obama's lead was cut to 3 percentage points, 46-43, with Nader getting 6% and Barr at 3%.

Note this comment though about polling for 3rd party candidates:

"A useful rule of thumb is that third-party candidates in November get no more than half the support polls show them having in June or July," Holland said.

Comments: The poll was taken over the weekend (favors Democrats) and surveyed registered, not likely, voters (favors Democrats).

 

[Puppycow]

There was some discussion about whether Barr might swing Georgia to Obama, but the current polls at least suggest not:
Georgia (RCP)

I think that among the various polls, the Gallup and Rasmussen tracking polls are probably the best because they are large samples.

The so-called "margin of error" seems to really be theoretical to me. I think polls are very often off by more than the margin of error.

 

[aerosolben]

I think that among the various polls, the Gallup and Rasmussen tracking polls are probably the best because they are large samples.

The so-called "margin of error" seems to really be theoretical to me. I think polls are very often off by more than the margin of error.

You already mentioned RCP, but here's a site I've been using lately: FiveThirtyEight. They do exactly what you're suggesting - examine historical performance of pollsters to determine their reliability and include that information when weighing averages. Plus some trend correction based on regression, demographics, and historical voting.

RE: The CNN Poll (I posted this in another thread before I saw this one, but it definitely belongs here too):

CNN seems to believe a 5-point lead (on 3.5-point margin of error) in their poll is a statistical dead heat.

"Every standard telephone poll taken in June has shown Obama ahead of McCain, with nearly all of them showing Obama's margin somewhere between three and six points," [CNN Polling Director Keating] Holland said. "In most of them, that margin is not enough to give him a lead in a statistical sense".

Sure, if by "most of them", you mean none of them.

Very odd - I wonder if they'll issue a correction. Or if I missed something, because they seem waaaay off.

 

[Brainster]

The so-called "margin of error" seems to really be theoretical to me. I think polls are very often off by more than the margin of error.

Well, the poll should usually be within the margin of error about 95% of the time. If it is not, there are three possibilities:

1. That one in 20 chance that 95% implies.
2. That public opinion moved after the polling (see New Hampshire).
3. That there was a problem with the methodology of the polls.

I like that 538 site as well, aerosolben; the only thing that worries me is that he admits to being an Obama supporter.

 

[UserGoogol]

There was some discussion about whether Barr might swing Georgia to Obama, but the current polls at least suggest not:
Georgia (RCP)

I think that among the various polls, the Gallup and Rasmussen tracking polls are probably the best because they are large samples.

The so-called "margin of error" seems to really be theoretical to me. I think polls are very often off by more than the margin of error.

Yes, it is theoretical. The margin of error is a purely mathematical concept based on the assumption that the poll is an statistically representative random sample of the population. Error introduced by problems inherent with the poll itself (people who answer telephone polls might not be representative, people lie to pollsters, the way the question is asked, etc) needs to be measured empirically and is harder to take into account.

 

[UserGoogol]

Well, the poll should usually be within the margin of error about 95% of the time. If it is not, there are three possibilities:

1. That one in 20 chance that 95% implies.
2. That public opinion moved after the polling (see New Hampshire).
3. That there was a problem with the methodology of the polls.

I like that 538 site as well, aerosolben; the only thing that worries me is that he admits to being an Obama supporter.

For the record, a 95% confidence interval does not actually mean that there's a 95% chance that the poll result is within the margin, but that if the actual proportion is within the interval then there would be a 95% chance of getting the poll result that was gotten. Which is not quite the same thing.

 

[Puppycow]

Well, the poll should usually be within the margin of error about 95% of the time. If it is not, there are three possibilities:

1. That one in 20 chance that 95% implies.
2. That public opinion moved after the polling (see New Hampshire).
3. That there was a problem with the methodology of the polls.

I like that 538 site as well, aerosolben; the only thing that worries me is that he admits to being an Obama supporter.

I think the problem with New Hampshire is maybe more complicated than just public opinion moving after the polls. First of all the polls were about as close to the election as you could get.

My theory is that since you had two primaries happening at once and independent voters could vote in either that there was a significant number of independent who were torn between voting for McCain and Obama, and they voted for McCain because the polls said that the republican primary was tighter.

If the polls had shown McCain winning by a mile and a tight race on the democratic side, maybe there would have been the opposite phenomenon?

 

[aerosolben]

I like that 538 site as well, aerosolben; the only thing that worries me is that he admits to being an Obama supporter.

Yes, I have considered that as a biasing factor, especially considering Obama has been doing better there than elsewhere (e.g., RCP).

However, I think the methodology used has been described quite robustly - even though I'm not sure if the full details have been published (i.e., sufficient to recreate the model yourself). The only weighting that isn't largely transparent from the main page is the 538 regression, but intermediate results are given. The benefits of superior process outweigh the potential for bias, in my mind. (especially considering the CNN incident mentioned above)

The blog, OTOH, certainly leans to the left when not discussing methodology.

 

[Puppycow]

McCain and Obama both perceived as flip-floppers:

More people, however, think that McCain "Would best handle the situation in Iraq"
48% to 38%. This is despite the war's general unpopularity. Also, 53% to 33% think that McCain "Would best protect the U.S. against terrorism."

Obama can take some solace in the fact that more people think that Obama is "Best able to handle the economy" 44% to 37%. He also creams McCain on "likability" 58% to 23%.

 

[aerosolben]

I like that 538 site as well, aerosolben; the only thing that worries me is that he admits to being an Obama supporter.

Just to follow up on this point; another concern I had with the site is the relatively frequent algorithmic tweaks had me somewhat worried he may be over-correcting to existing data rather than identifying the true underlying process.

However, today's blog post pointed out that the blog author, Nate Silver, is also the creator of PECOTA, one of the more successful and accurate systems in the overcrowded field of sabermetrics (I probably should have noticed when he initially revealed his identity a month ago, but...). This reassures me a good deal on his statistical credibility.

 

[Brainster]

Rasmussen Reports on Missouri:

The latest Rasmussen Reports telephone survey in Missouri shows John McCain attracting 47% of the vote while Barack Obama earns 42%. A month ago, the candidates were essentially even. That survey was conducted the night that Obama clinched the Democratic Presidential nomination. McCain had the advantage in earlier surveys.

McCain wins the favorable/unfavorable ratings in Missouri as well, with 58/39 compared to Obama's 50/48.

Date of Poll: 7/7/08
Respondents: 500 likely voters
Error Margin: +/- 4.5%

Missouri is considered a bellwether state; the only candidate to win Missouri in the last 100 years who did not win the presidency was Adlai Stevenson in 1956.

 

[aerosolben]

Missouri is considered a bellwether state; the only candidate to win Missouri in the last 100 years who did not win the presidency was Adlai Stevenson in 1956.

Ugh. I hope you're merely repeating the public perception of Missouri and not endorsing it - I hate the "bellweather" state argument. A state is a bellweather until it isn't - it's the worst sort of post-facto justification. Maine used to be a bellweather - no one seems to think Obama will win by 10 points, though. (And its not like Missouri "just missed" in 1956)

If there's a REAL bellweather state for this particular election, it's Ohio.

 

[Brainster]

Ugh. I hope you're merely repeating the public perception of Missouri and not endorsing it - I hate the "bellweather" state argument. A state is a bellweather until it isn't - it's the worst sort of post-facto justification. Maine used to be a bellweather - no one seems to think Obama will win by 10 points, though. (And its not like Missouri "just missed" in 1956)

If there's a REAL bellweather state for this particular election, it's Ohio.

Actually Missouri did "just miss" in 1956; Stevenson got 50.1% and Eisenhower 49.9%. The vote difference was just a hair under 4000 votes.

It's reasonable to point out that there are bellwethers (note the correct spelling). Missouri makes sense as one; it's a swing state, it's centrally located, with Midwest influences and Southern influences. When as simple an indicator as one state has been proven right 25 of the last 26 elections, it's not hard to see that it's no coincidence. There are other indicators that say McCain will lose even if he wins Missouri; no Republican has won without Ohio, for example, which McCain is losing right now.

But if McCain actually wins Missouri, I'd be very, very surprised if he doesn't win the presidency. And if McCain loses Ohio, I'd be pretty surprised if he wins the presidency. I'm going to guess that he either wins both or loses both to continue the trend.

 

[aerosolben]

Actually Missouri did "just miss" in 1956; Stevenson got 50.1% and Eisenhower 49.9%. The vote difference was just a hair under 4000 votes.

I meant the national election as a whole; Stevenson was not exactly in the running. That's a fair point, though.

It's reasonable to point out that there are bellwethers (note the correct spelling).

Noted - had a brain fart for some reason.

Missouri makes sense as one; it's a swing state, it's centrally located, with Midwest influences and Southern influences. When as simple an indicator as one state has been proven right 25 of the last 26 elections, it's not hard to see that it's no coincidence.

Yes, but the odds are currently that Missouri will not pick the winner (obviously, things can change, etc). If that's the case, people will simply find a new bellwether for the next election.

I'm not saying moderate states can't run up streaks, I just think it's simply more meaningful to look at current political views and base predictions around that - shifting demographics and party alignments have made short work of many historical bellwethers (see Maine). Relying on historical success is foolhardy when they disagree with more modern information is available - does the 1908 election really have predictive relevance?

There are other indicators that say McCain will lose even if he wins Missouri; no Republican has won without Ohio, for example, which McCain is losing right now.

Ohio is a legitimate bellwether because it is backed by current national opinion. It is on the tipping point of national opinion - if it shifts to the right far enough that Ohio goes red, McCain would probably have enough votes to take the election and vice versa.

But if McCain actually wins Missouri, I'd be very, very surprised if he doesn't win the presidency.

Using the projections from fivethirtyeight.com, I get 343 electoral votes to which are "more blue" than Missouri (versus 284 for Ohio), which is fair margin to lose your bet in. Now, there may be some regional variation prior to the election which disrupts that arrangement, but there's no reason to believe it will move Missouri in any particular direction on the scale.

 

[Brainster]

My point is more that if, on election day, Missouri goes for McCain, it will probably indicate that the country as a whole has moved towards the senator from Arizona. And if the country as a whole has not moved towards McCain, then he will probably not win Missouri. By mentioning this in connection with July poll results I have made this unclear. And certainly as Maine shows, you are correct that all trends continue until they stop.

Rasmussen has some additional polling numbers. Obama leads McCain nationally:

The Rasmussen Reports daily Presidential Tracking Poll for Thursday shows Barack Obama attracting 45% of the vote while John McCain earns 41%. When "leaners" are included, Obama leads 48% to 45%. This is the first time that McCain has been within three points of Obama with or without leaners since June 20.

Date of Poll: 7/8-7/10/08
Sample Size 3000 likely voters (three day 1000 LV rolling average)
Margin of Error: +/- 2%

Obama still leads in New Jersey, but the race appears to be tightening there as well:

The latest Rasmussen Reports telephone survey of New Jersey voters, taken Monday night, shows Obama ahead of Republican candidate John McCain 44% to 39%. Last month, the Democratic hopeful had a 48% to 39% lead over his GOP rival

Date of Poll: 7/7/08
Sample Size: 500 likely voters
Margin of Error: +/- 4.5%

In Wisconsin, a very close state in 2004, Obama's opened up a pretty good gap:

The latest Rasmussen Reports telephone survey in Wisconsin shows Barack Obama earning 50% of the vote while John McCain attracts support from 39%. This is a big improvement for Obama who held a statistically insignificant two point lead in the state a month ago. Last month’s poll was taken just before Hillary Clinton dropped out of the race. When “leaners” are included in the current totals, it’s Obama 52% and McCain 42%.

Poll Date: 7/8/08
Sample Size: 500 likely voters
Margin of Error: 4.5%

Obama is leading his home state of Illinois (not exactly a shock):

Senator Barack Obama leads John McCain 50% to 37% in his home state of Illinois, according to the latest Rasmussen Reports telephone survey in the state. When “leaners” are included, Obama leads 52% to 41%.

Poll Date: 7/8/08
Sample Size: 500 likely voters
Margin of Error: 4.5%

 

[aerosolben]

My point is more that if, on election day, Missouri goes for McCain, it will probably indicate that the country as a whole has moved towards the senator from Arizona. And if the country as a whole has not moved towards McCain, then he will probably not win Missouri. By mentioning this in connection with July poll results I have made this unclear.

Actually, that's true if you substitute Obama, since Missouri is leaning to the right relative to the rest of the country (and there is a good chance McCain loses with Missouri - that is actually the current forecast). My counterpoint is that the states smack in the middle of the left-right spectrum (based on current information) are the better bellwethers.

And certainly as Maine shows, you are correct that all trends continue until they stop.

And that's my real point - I don't have a problem with bellwether as long as the appropriate data is used to determine it. Historical information isn't woo, per say, simply subpar.

 

[SezMe]

Clever thread title, Brainster.

 

[David Wong]

So at what point in previous election cycles have we actually started to see, you know, movement?

We've had the odd poll showing Obama up by 12 but they all fall into place eventually, with Obama up by 3-6 points. There's been no real trend in weeks... despite all the news stories and "gotcha" that's come and gone since then.

So at what point do we actually start to see movement? The debates?

I know it's summer and nobody cares yet, so at what point is it worthwhile to tune back in? Is it like the NBA where there's no point in tuning in until a month before the finals?

 

[SezMe]

After the conventions.

 

[Brainster]

Something for both sides:

CNN sees Obama's margin widening:

In the latest CNN Poll of polls, the Illinois senator holds an 8-point lead among registered voters nationwide, 49 percent to 41 percent, with 10 percent still undecided.

The 8-point spread is up 2 points from a CNN poll of polls released one week ago. It is double the margin it was one month ago, days after the prolonged Democratic race ended.

This appears to be like the RCP average, not a poll itself, but an amalgamation of several polls. RCP sees it as a closer race, with only a 4.6 percentage point margin for Barack.

On the other hand, Newsweek now sees it as a tossup (Obama by 3 points), after a week earlier pointing to a blowout for Obama:

A month after emerging victorious from the bruising Democratic nominating contest, some of Barack Obama's glow may be fading. In the latest NEWSWEEK Poll, the Illinois senator leads Republican nominee John McCain by just 3 percentage points, 44 percent to 41 percent. The statistical dead heat is a marked change from last month's NEWSWEEK Poll, where Obama led McCain by 15 points, 51 percent to 36 percent.

Date: 7/9-7/10/08
Sample Size: 1037 Registered Voters
Error Margin: +/- 3%

Of course, that large a swing does not seem justified by any news events, and the prior poll was way out of line with all other polls.