Questioning The Questions

February 6th, 2012
08:50 PM ET

About the WaPo/ABC News poll referenced in tonight's show
In tonight's broadcast, we reference a WaPo/ABC poll that suggests Romney's support in the November general election is waning. The Romney campaign has publicly questioned the methodology behind the poll. Our guest, Benjamin Ginsberg, a senior Romney campaign advisor, called it into question on air with us. To be fair, we think ABC News and the Washington Post should have a chance to defend themselves. Below, we've pasted their explanation of the poll and included a link to their site.

Today’s Poll: A Word on the Buzz
- by Gary Langer
http://abcnews.go.com/blogs/politics/2012/02/todays-poll-a-word-on-the-buzz/

Our latest ABC News/Washington Post poll is getting some buzz today, including some criticisms of the questionnaire design. On one hand it’s hardly the newest game in town for aggrieved parties to try to dismiss survey results they don’t like. On the other, fair-minded discussion always is welcome.

We pride ourselves on best-practice survey methodology based on trained interviewers calling randomly dialed cell and landline phone users. We strive for neutral, balanced questions and independent analysis. We try to answer all inquiries about our data, and we release our questionnaires and ultimately our raw datasets for all comers to review.

The poll we’ve released today included balanced questions on recent issues involving Mitt Romney’s wealth, taxpaying and business background. Each was neutrally presented – asking, for instance whether he “is or is not paying his fair share of taxes,” whether he “‘achieved the American dream” or “benefitted from opportunities that are not available to other people,” and whether he did more to “create jobs” or to “cut jobs” at Bain Capital. The parallel phrases in the last two were asked in rotated order.

Critics today have suggested that asking these questions before the general election horse race may have biased its results. There are reasons to think otherwise.

First, a different question immediately preceded the vote question, one testing issues involving three candidates – Romney, Newt Gingrich and Barack Obama. One of those turned out to be strongly positive for Romney, measuring views of his business experience. We have trend for this question among Republicans and Republican-leaning independents; their views of Romney’s business experience were no less positive in this poll than they were in December, with different preceding questions.

Second, the logic that Romney’s wealth, tax history and record of job creation are negatives and suppress his support, could work just as well in reverse. Asked neutrally, they are neither negatives nor positives, but simply salient attributes on which public attitudes matter.

Third, key questions in an incumbent election come far earlier in the questionnaire. Our very first question found Barack Obama with a 50 percent job approval rating, his highest since last spring. A subsequent question found 50 percent saying he deserved re-election. It seems unsurprising that later we found 51 percent preferring Obama over Romney in a head-to-head-matchup.

Indeed these are of a piece. Among people who approve of Obama’s job performance, 91 percent prefer him over Romney; among those who disapprove of Obama, 88 percent prefer Romney for president. In our previous three polls, in January, December and November, Romney won 88, 84 and 77 percent support, respectively, from Obama disapprovers; Obama won 88, 87 and 90 percent support from his approvers. Those results make suggestions of order-effect in this survey look like a tough sell.

It’s also perhaps worth noting the long line of surveys we’ve put out that had much worse numbers for Obama – down to 42 percent approval last October, for instance – and his Democratic Party, whose hammering in the 2010 midterm elections was correctly anticipated in our pre-election polling. His side may have resented those results. Not our problem.

Romney, for his part, may not love that 66 percent of Americans don’t think he’s paying his fair share of taxes. On the other hand, he may brighten up at the fact that many more see his business experience as a major reason to support him rather than as a major reason to oppose him – with this item asked as part of the question that directly preceded vote preferences. Again, pleasing any candidate is not our concern.

Lastly, we’ve no need to hide behind the work of others, but Obama led Romney by 6 points in an NBC News/ Wall Street Journal poll a few weeks ago, the same margin as in our survey (two others recently had them tied) and by 10 points in a UNH poll out of swing-state New Hampshire just last week.

There’s plenty of time in the election year ahead, and plenty of measurements to make. We tend to focus less on the horse race and more on underlying public attitudes about the issues and candidate attributes. From that perspective we think today’s survey tells a useful, independent, unbiased story about the political landscape. We’ll keep at it, while welcoming continued open discussion along the way.