At Catallaxy, Rafe pings Club Troppo for getting "excited by a report from the US which suggested that a large proportion of Republican voters have really silly ideas, indeed they are practically insane. Interesting to read that this result came from a survey commissioned by Daily Kos and they have now admitted that the survey results are bogus."
What’s this about?
According to poll results posted on Daily Kos: 63% of self-identified Republicans believe that Obama is a socialist, 42% believe he was not born in the United States, 31% believe he is a racist who hates White people and 31% think contraceptives should be outlawed. After looking over the results Markos Moulitsas said :
… I expected some of these questions to generate some pretty crazy responses. But I gotta say that it sorta exceeded all, all my expectations. I mean you have a third that’s just certifiably insane. You have a third that’s fairly reasonable. And generally you have a third that thinks it’s debatable whether Obama’s a socialist. Thinks it’s debatable whether Obama’s a racist who hates white people. And so on and so forth. So I don’t know if I quite expected just how crazy these results were going to be.
At the time, Moulitsas was working on a book called American Taliban. The book argues that American conservatives share the same agenda as the Islamic world’s radical Jihadists. "I found myself making certain claims about Republicans that I didn’t know if they could be backed up", wrote Moulitsas. "So I thought, ‘why don’t we ask them directly?’ And so, this massive poll, by non-partisan independent pollster Research 2000 of over 2,000 self-identified Republicans, was born."
The results generated excitement and more than a little scepticism. At Atlanticwire Michael Kinsley noted that "Even some on the left believe the poll also had structural deficiencies." Debate raged over the effects of sampling bias, response rates, and acquiescence bias. But at the time almost everyone assumed that the poll was legit. Earlier this week that assumption collapsed when Moulitsas announced that a year and half worth of polling results were probably “bunk”.
Moulitsas is planning to sue his polling company Research 2000 after a group of independent researchers contacted him and explained why the polling results looked suspicious. "Based on the report of the statisticians, it’s clear that we did not get what we paid for", Moulitsas wrote. "We were defrauded by Research 2000, and while we don’t know if some or all of the data was fabricated or manipulated beyond recognition, we know we can’t trust it."
After this disaster unfolded, I rushed back to the manuscript to see if there was any collateral damage. Turns out the entire book has only two citations to Research 2000 polling, and both of those times, the results were backed up by additional numbers from other polling outfits. The first was backed up by the right-wing Rasmussen, the second by Pew.
At Daily Kos Moulitsas links to other polls that report similar findings. One is this Harris Interactive poll that reported 67% of Republicans believe Obama is a socialist and 45% that he was not born in the United States and so is not eligible to be president. Critics were quick to pick holes in the poll’s methodology and design.
Even if Moulitsas’ pollster had played by the rules, it would be difficult to know what conclusions to draw. Moulitsas wants us to believe that Republicans carry these attitudes around in their heads, think about them frequently and use them to inform political decisions. But none of this is established by the polls he refers to.
A popular myth about polling is that it uncovers already existing attitudes. But in many cases the respondent hasn’t thought about the issue before the pollster confronts them with a question. As I explained in an earlier Troppo post, it’s often possible persuade respondents to volunteer attitudes to fictitious issues such as non-existent laws. Polls don’t tell us whether the person had a particular attitude before they were asked about it, how much time they spend thinking about it, what they think it means or how it affects their behaviour.
The Pew Research Center for People and the Press recently reported that 41% of American adults believe that Jesus Christ will definitely or probably return to earth before 2050. Respondents from the South, those with less education and White evangelicals were more likely to answer definitely or probably than other groups. How many of you think this has bankable implications for companies selling retirement savings plans or durable building materials?