Kos charges polling fraud

[Dancing David]

Essentially 538.com says that the data do not fluctuate enough, as would be expected from a random sample.

So either they sampled a very small pool, say the same 400 people, or they selected the pool from a homogenous group.

 

[Puppycow]

What I can't see is a way to get these kind of results if you are making up numbers (simplest method pic the number you want then add noise).

Some function that starts with one number and then adds the same integer to men that it subtracts from women (or vise versa)?

Not sure why a faker would pick such a method though.

ETA:
For example, first he picks the overall number (total of men plus women), and then he picks the spread between men and women, which he makes an even number for simplicity's sake, and then divides it in half and adds it to either men or women and subtracts it from the other.

 

[Beerina]

Essentially 538.com says that the data do not fluctuate enough, as would be expected from a random sample.

So either they sampled a very small pool, say the same 400 people, or they selected the pool from a homogenous group.

Or they faked it.

 

[Dancing David]

Or they faked it.

Sure I was providing alternatives, I would like to see their sample verification on the phone calls.

 

[Brainster]

Some interesting speculation on the blogs today about whether Kos should have realized he was getting scammed. NextRight's Patrick Ruffini notes that a good polling company would have been charging him much more than Kos maintains is his operating cost:

Now, even weirder, it is just of Republicans, AND it wasn't done from a listed sample. When you want to do a survey of, say, primary voters in a statewide, a listed/voter file sample is a totally acceptable practice because the alternative is unbelievably costly. Think about it - not only did they call 2000 people, but they randomly dialed people, and turned away anyone who didn't identify as a Republican. This will crush your incidence rate (meaning the number of folks who pick up the phone who are eligible to take the survey) and send costs through the roof. We're talking at least tripling the costs.

A survey of the length of that January 2010 survey, about 25 short-ish questions, plus a handful of demographics, is probably about a 10 minute questionnaire (I'm just eyeballing it and assuming an introductory statement and guessing on the # of demos asked, I could be off by a few minutes). Fielding a 10 minute questionnaire to 1000 registered voters is going to run you in the $25-30 range. Fielding a 10 minute questionnaire to 2000 voters? Probably 45-55. But with the crazy drop in incidence caused by the Republican screener? That survey could not have been done for less than six figures. Period.

Obviously the dollar amounts quoted above are intended to be thousands; the final estimate confirms that.

Obviously Ruffini is a conservative, although generally a fair one. But Nate Silver is not, and he talks about another smoking gun: The discount Kos got for paying R2K early:

Say that it cost Research 2000 about $4 per interview to complete the 52 weekly, State of the Nation polls that it had agreed to provide to Daily Kos, which at the time consisted of 2,400 adult respondents each. Please note that the cost estimates included here are hypothetical. Based on my limited experience in actually commissioning polls, this would have been quite cheap for traditional telephone polling, but we'll run with it for demonstration purposes in the absence of other evidence; it works out to $499,200 over the course of the year. Say also that the 98 state- and district-level polls that Research 2000 had originally agreed to provide to Daily Kos in 2009 cost it $6 per interview; the higher cost reflects the fact that these were polls of likely voters and therefore would have been more expensive to complete, since it takes time to screen the unlikely voters out from your sample. With 600 respondents per local poll, this would have cost Research 2000 an additional $352,800 over the course of the year.

Thus, the total cost of Research 2000's polling in 2009 would have been $852,000 -- $499,200 for the national poll, and $352,800 for the local polls. Suppose that 40 percent of the fees that Daily Kos owed Research 2000 for this polling were due with the third and final payment -- this would have been $340,800.

In exchange for receiving this $340,800 a few months earlier than it otherwise would have, Research 2000 was willing to conduct 59 additional polls for Daily Kos. The cost of these polls, assuming they were 600-person state and local polls conducted at a cost of $6 per interview, would have been $212,400.

So both Ruffini and Silver agree that Kos should have been paying enormous sums of money to get the kind of polling that Research 2000 claimed to be providing. And yet per Ruffini Kos generates gross revenues of about a million total.

It's hard for me to believe that Kos's polling bills wouldn't have run into the deep six figures, which seems like an awfully big chunk of his $1 million (give or take) in revenue.

Note as well that Kos admits in his lawsuit that he never had a formal contract with R2K. If as Silver estimates, he was paying $750,000 annually for R2K's services, this is an astounding business practice. If, on the other hand, he was paying far less, it seems obvious that he should have heeded the old adage of, "You get what you pay for."

 

[GreNME]

That doesn't seem worth much, really. I mean, wasn't the whole point of commissioning the poll to validate Kos's preconceived notions about conservatives?

Removing references to the book leaves behind the preconceived notions, only presented as bald, unsupported assertions, rather than rational, evidence-based conclusions... And meanwhile everybody now knows that the evidence upon which those assertions are based, and around which the book was written, is bogus.

Unless he's going to entirely re-write the book, referencing reputable polls and reaching conclusions based on real data, I don't really see how it can be anything other than a dog's breakfast.

In fact, deleting the references but keeping the claims seems pretty slimy to me.

Wow, care to substantiate any of that? That seems to not be at all what was stated in the link you were responding to.

So, whence all the assumptions?

 

[theprestige]

Wow, care to substantiate any of that? That seems to not be at all what was stated in the link you were responding to.

So, whence all the assumptions?

I'm pre-testing for the MDC. We'll see how noticeable the deleted references are once the book comes out.

 

[Puppycow]

Note as well that Kos admits in his lawsuit that he never had a formal contract with R2K. If as Silver estimates, he was paying $750,000 annually for R2K's services, this is an astounding business practice. If, on the other hand, he was paying far less, it seems obvious that he should have heeded the old adage of, "You get what you pay for."

Maybe he was relying on his "intangible right to honest services."

 

[Brainster]

Maybe he was relying on his "intangible right to honest services."

Well, I'm certainly not defending R2K by any means. But if, as it seems increasingly likely, Kos was getting supposed honest services at a ridiculously low price, it does raise questions in my mind.

The problem that Kos ran into is that Silver's blog raised some very difficult questions about R2K's reliability. In particular, Research 2000 had issued a poll showing Blanche Lincoln losing her Senate seat in a primary run-off in Arkansas. Because the poll confirmed the meme that incumbents were in trouble in 2010, the media ran with it, and (more important) the unions and netroots redoubled their efforts to defeat her and nominate Bill Halter. But in the end she won by four percentage points, in what the media proclaimed a stunning upset. Why was it an upset? Because of the polls provided by R2K. Incidentally, the White House was pretty pissed, noting that the unions had blown $10 million on Halter's campaign.

Now here's the really tough part. Kos and many of the other netroots bloggers hate Blanche Lincoln, and so they had a vested interest in providing polls that showed her losing to Halter. So now the question becomes whether there was collusion between the polling organization and the people ordering the poll to come up with the desired result. And at that point, the poll really shouldn't cost much at all.

 

[GreNME]

I'm pre-testing for the MDC. We'll see how noticeable the deleted references are once the book comes out.

You get points for bringing teh funnay. However, it sort of undermines the crux of your (and Brainster's) criticisms, does it not?

 

[hgc]

...
Now here's the really tough part. Kos and many of the other netroots bloggers hate Blanche Lincoln, and so they had a vested interest in providing polls that showed her losing to Halter. So now the question becomes whether there was collusion between the polling organization and the people ordering the poll to come up with the desired result. And at that point, the poll really shouldn't cost much at all.

It doesn't seem too likely that Kos colluded intentionally with R2K over Halter/Lincoln, considering that Kos has made a very embarrassing admission about his own realiability in blowing the cover off of R2K so publicly. As for subconscious collusion in support of wishful thinking, that's not so far-fetched. And yeah, if he was getting such an abundance of polling for a rock-bottom price, he should have asked himself why.

 

[theprestige]

You get points for bringing teh funnay. However, it sort of undermines the crux of your (and Brainster's) criticisms, does it not?

Yes. Yes it does. Now what?

 

[Puppycow]

First look at Daily Kos's lawsuit: Charges Research 2000 with "fraudulently manufacturing phony results"

I've obtained a copy of the lawsuit that Daily Kos just filed against Research 2000, and this going to get nastier than you thought.

The suit contains striking new details about Research 2000's alleged reluctance to release its raw data and its alleged money problems, directly alleging that the firm committed deliberate fraud by selling DailyKos years of data that was "phony" and "falsified."

Daily Kos's lawyer also tells me that he has sent a letter to the firm's attorney informing them they have a legal obligation to preserve all info relevant to the case -- such as computer files holding raw polling data and phone records documenting the polling calls. "In due course, we'll file the appropriate discovery requests to obtain the relevant information to which my clients are entitled," Daily Kos's lawyer, Adam Bonin, tells me.

However, the lawsuit does not say how much the total damages are except that they are in excess of $100,000.

The damages claimed by Daily Kos are likely to be much higher than $100.000, once punitive, reputational and full actual damages are claimed. Daily Kos merely declared that number because the court requires damages to be in excess of that sum for the suit to proceed.

Also: The suit is alleging only that the weekly polls -- not other polls, such as the battleground horse race surveys -- were fraudulent, further rendering the cost question raised above irrelevant.