Poll Fight!

[ICantSpellDawg]

RCP selects major polls and combines them together by population. Are you suggesting that RCP is using uncouth methods to combine polls? Are they part of the vast right-wing conspiracy do you think?

The reality is that you would prefer a polling site maintained by Obama supporters and I support a polling site maintained by centerist guys. I don't detect a bias in RCP averages, but I wouldn't because I am par of the Republican hate machine.

538 has a faq. Hurry up RCP, close up shop because you've been sniffed out!

What more do you need? RCP combines major polls by population and gets averages. How hard is that? What polls!!!? Where do they come from!!!?

Looking closer into the sites, 538 has biased editorial accompanying their polls whereas RCP has very little editorial anywhere near the polls. Jay Cost is a mastermind and is a major draw to RCP. Only time will tell if 538 has a mind like that on staff. Also, 538 says that they "handicap" polling averages within the larger average based on their subjective view of accuracy in polling historically. They claim this, but I don't see an explanation as to what qualifies polling as accurate or not before an election. Maybe I just missed it, but I'd like to see what each poll's handicap is and how they were rated.

[Lemur]

538 throwing out a bunch of claims doesn't really prove anything at all does it? Just because they make a bunch of charges that may or may not have any factual basis, the RCP team is obligated to respond?

RCP selects major polls and combines them together by population. Are you suggesting that RCP is using uncouth methods to combine polls? Are they part of the vast right-wing conspiracy do you think?

I agree with 538 this far: RCP does not disclose its methodology, certainly not in a way that would allay any worries about statistical shenanigans. As for RCP being part of the Vast Rightwing Conspiracy, I'll help out if you need some more straw to stuff in that man you're building.

[Xiahou]

I agree with 538 this far: RCP does not disclose its methodology

Neither do many of the polls that both entities average. That doesn't make any claims of bias against these polls automatically valid.

In fact, as I've said before, I really don't think averaging polls like this has any real statistical validity- so it's kind of hard to get worked up about it either way.

[Don Corleone]

I don't know enough to say whether RCP is in the tank for McCain as 538 is alleging or not.

But I do know business. If you're a young, up-start up and comer in an IP-based business, it is to your distinct advantage to learn how the successes in your industry make their secret sauce. Sure, they disclose their own methodology, because they know it's not sought after.

This is the equivalent, to me, of an open-source developer getting pissed at Microsoft for not disclosing the source code for a new killer app.

[m52nickerson]

I don't know enough to say whether RCP is in the tank for McCain as 538 is alleging or not.

But I do know business. If you're a young, up-start up and comer in an IP-based business, it is to your distinct advantage to learn how the successes in your industry make their secret sauce. Sure, they disclose their own methodology, because they know it's not sought after.

This is the equivalent, to me, of an open-source developer getting pissed at Microsoft for not disclosing the source code for a new killer app.

Being an Obama supporter even the notion that RCP shows favor to McCain is more reason to go to that site. If Obama is leading in the polls on a McCain+ site that means to me Obama really is leading.

[Lemur]

But I do know business. If you're a young, up-start up and comer in an IP-based business, it is to your distinct advantage to learn how the successes in your industry make their secret sauce. Sure, they disclose their own methodology, because they know it's not sought after.

This isn't quite applicable -- RCP does reveal which polls they include. They just don't say why. Anywhere. What set 538 off was the inclusion of a notably flaky poll that suddenly favored McCain at a time when Obama's numbers are on an upswing. It was suspicious, and without any rationale or logic from RCP, it looks odd.

ARG had been effectively "banned" from RCP for several months now, going back to the Democratic primaries. Sometimes RCP listed ARG polls with an asterisk and did not include them in their averages; more often they didn't list them at all.

Why didn't RCP include ARG's polls in its averages? Presumably because they had an exceptionally erratic performance during this year's primaries, a performance that -- when coupled with ARG's relative lack of disclosure about its methodology -- had led RCP to conclude that their polls were not credible. [...]

What I do know, however, is that whatever RCP's reasons were for excluding ARG from its averages, none of those reasons should have changed over the past 48 hours. Did ARG suddenly improve its level of disclosure? No. Did their polls suddenly become more accurate? How could they, when we haven't had any elections to evaluate them.

No. The only thing that changed is that ARG released a set of three polls yesterday that had considerably more favorable numbers for John McCain than other contemporaneous polls of those states. In each of those states -- North Carolina, Virginia, Nevada -- ARG is presently the outlier on the John McCain side (tied with Mason-Dixon in the case of Virginia). RCP did not feel any compulsion to include ARG's numbers when ARG cycled through all 50 states a couple of weeks ago (including many where there has been very little polling). Only when ARG released this gravity-defying set of polling in Virginia and North Carolina and Nevada did RCP suddenly have a change of heart.

[ICantSpellDawg]

This isn't quite applicable -- RCP does reveal which polls they include. They just don't say why. Anywhere. What set 538 off was the inclusion of a notably flaky poll that suddenly favored McCain at a time when Obama's numbers are on an upswing. It was suspicious, and without any rationale or logic from RCP, it looks odd.

ARG had been effectively "banned" from RCP for several months now, going back to the Democratic primaries. Sometimes RCP listed ARG polls with an asterisk and did not include them in their averages; more often they didn't list them at all.

Why didn't RCP include ARG's polls in its averages? Presumably because they had an exceptionally erratic performance during this year's primaries, a performance that -- when coupled with ARG's relative lack of disclosure about its methodology -- had led RCP to conclude that their polls were not credible. [...]

What I do know, however, is that whatever RCP's reasons were for excluding ARG from its averages, none of those reasons should have changed over the past 48 hours. Did ARG suddenly improve its level of disclosure? No. Did their polls suddenly become more accurate? How could they, when we haven't had any elections to evaluate them.

No. The only thing that changed is that ARG released a set of three polls yesterday that had considerably more favorable numbers for John McCain than other contemporaneous polls of those states. In each of those states -- North Carolina, Virginia, Nevada -- ARG is presently the outlier on the John McCain side (tied with Mason-Dixon in the case of Virginia). RCP did not feel any compulsion to include ARG's numbers when ARG cycled through all 50 states a couple of weeks ago (including many where there has been very little polling). Only when ARG released this gravity-defying set of polling in Virginia and North Carolina and Nevada did RCP suddenly have a change of heart.

It really sounds like you are stretching this thing. I'm glad that you've brought 538 to our attention. It is one more site to bounce numbers off of. Maybe I'll average RCP with 538 and be sure to disclose my methodology!