Your May mailbag questions, answered. "Ask Away" is here!
On voter enthusiasm, the Handmaid's Tale, and who honestly has the time and energy to take a poll these days.
Welcome to Codebook, a newsletter that decodes our world through polling and research.
This edition—"Ask Away!"—is a monthly feature rounding up the best reader questions.
Please subscribe here and follow my page on Facebook for the latest. And don't forget: Every Monday is a premium subscriber-only roundup of the numbers you need heading into the week.
It's the month of May, which means beautiful weather, great motorsports action (Indy 500! Monaco!) and a lot of exciting political primaries. People are beginning to care about polls again, which is both lovely and keeps me busy.
Thank you to everyone who submitted a question this month! As a reminder, if you want to get a question in for a future Ask Away, there are a few ways to do it.
1. Comment on Bulletin under any post. I see all the comments!
2. Comment on a post on my Kristen Soltis Anderson writer Facebook page or send a message to that profile. I read your notes!
3. Reply to me on other social media platforms (Instagram and Twitter: @ksoltisanderson).
4. Send me a Will Buxton Cameo.
On to your questions!
***
First question is from my high school friend (Go Bears!) Klarika who asks:
Who is the pool typically comprised of when there's polling? I don't know many people these days who have the time, let alone patience, to participate in a meaningful poll...I feel like an ass saying this to you, but I have a hard time putting stock into a lot of polling results because I feel like the only people who have the time to participate in such things are either (1) people who are avoiding getting back to work after lunch (Exhibit A being myself right this minute) or (2) keyboard warriors who have no real responsibilities and all the time in the world. Obviously that's not the case all the time, but I always want to see demographics of those polled, which I know is super impractical.
Hello Klarika! This is an excellent question in the world of mostly opt-in web panel polling. Telephone polling - formerly the industry standard - is now increasingly a tool that is mostly used by political pollsters and not market researchers. The upside of these web panels is people can take them any time, anywhere. You aren't just reaching "folks who happen to be home and near a landline phone at dinnertime." The downside is, who honestly has any time to take surveys?
Pew Research Center has been tracking their overall response rates for decades and has found that phone surveys have seen huge declines since the 1990s. (This has been exacerbated by the rise of cell phones, where people are more easily able to just screen out calls they don't know.)
But what is kind of wild is that even as response rates have fallen...polls have still been reasonably good? People think about things like the 2016 election and say all polls are broken, but then you get polls in the Ohio Republican Senate Primary last week that actually got things pretty close. If the only people taking polls were truly unrepresentative weirdos, you'd wind up with a result nowhere close to reality. So someone out there is taking polls, and those someones aren't as weird as you might think!
There's a conference happening right now (AAPOR, the American Association for Public Opinion Research) where researchers are diving into all the ways you can try to tackle the problem you identify, Klarika. Making sure, for instance, that you have a sample with the right proportion of people who own homes, or who say they voted for a particular candidate in the last election, etc. are all approaches different pollsters are trying to make sure they've got as representative a sample as possible.
The other thing pollsters should be doing is making sure a survey is clearly written and easy to take. If you're fielding a survey online, it needs to be programmed so someone taking it on a mobile device can take it just as easily as someone on a laptop or desktop. Your questionnaire shouldn't be too too long. (I am militant about this with my own clients, sometimes to their frustration.) I wrote about this in my March mailbag, referencing the 160+ question surveys YouGov US fields, noting that I generally am very pro-YouGov but I cannot for the life of me fathom who on earth has time for this.
Pollsters also throw in safeguards that should prevent you from, on your lunch hour, mindlessly clicking through a survey without paying attention. We regularly delete responses from people who take the survey much much faster than the median respondent, or who "straight-line" (meaning they just pick the first option for everything), and so on. We usually include a trap question or two ("Please choose 'somewhat unfavorable' for this question.") to make sure people are paying attention. It is not perfect, but it helps!
All of which is to say: you are not an ass for asking this and for questioning the polls. No one questions the polls more than pollsters themselves, trust me. Which leads me to my next question...
From another longtime friend, Matt H. of Florida:
If you could wave a magic wand and change one thing either about the polling industry or media coverage of polling, what would you change?
Last month I tackled a question about what I'd do if I could change polling methods with a magic wand, so this month I'll take the media coverage piece. Mostly, I wish we were better at conveying uncertainty without it sounding boring or wishy-washy.
A wise man once said to me "Polls are good for measuring pounds, not ounces." If Joe Biden's job approval goes from 41 percent to 43 percent, that's....somewhat interesting but not necessarily indicative of anything significant? You want a lot of different polls all moving in the same direction before you can start making sweeping conclusions. But no one is going to click on a headline that says "Poll says Biden might be improving with the public, might not be."
This is especially challenging on TV. I love going on TV to talk about the polls, but I never will have the same amount of time in a TV hit that I used to have on my old polling podcast, The Pollsters, or even the amount of time I get to yammer at the start of my SiriusXM radio show each week (Saturdays, 10 am, POTUS channel 124!). So I have to strike a balance between saying something that will inform and engage quickly without overstating what we do or don't know.
This means that some pundits can confidently declare things like "Policy X will be VERY unpopular with the public and shift the elections toward the Democrats" while you've got me out here saying "Well, the polling on this is complicated, and a lot depends on question wording" and so on. It's less punchy, but it's usually more accurate. I do the best I can!
Next question comes via Instagram Stories from Josh G.:
Are the Dems doomed in the midterms unless they pass major legislation?
I think whether or not Democrats are "doomed" at this point has very little to do with whether they pass major legislation in the next few months. Bear in mind, a major bipartisan infrastructure bill was passed during this past Congress. Infrastructure is super popular! And yet.
The national political environment is one that is just bad news for Democrats in general. Part of this is because inflation is a serious problem and Democrats are the party in power while people's cost of living is soaring. Part of this is just the challenge of being the party in power at all in a midterm. Unless Congress could wave a magic legislative wand and make gas go back to $2.50 a gallon, make sure all shelves are stocked, bring housing costs down and so on, all within the next six months...it's going to be tough.
But I'll present two contrasting views on this. The first is a tweet I saw from Cook Political Report's Amy Walter this week reflecting on a focus group of "infrequent Dem voters." Her observation was that these voters feel depressed and want to see executive action.
I am not one to doubt Amy's analysis, and (at least pre-Roe drama) Democrats did have a bit of an enthusiasm gap, but I'm just not sure that executive orders will move the needle as much as this focus group participant claims.
The flip side of all of this is from Ruy Teixiera, a prominent analyst on the Democratic side. He, too, is critical of Democrats for not getting a whole lot done, and places the blame squarely on the party's left flank for pushing for things that were just unrealistic (especially with a 50-vote "majority" in the Senate in the age of The Manchin-Sinema Supremacy). Instead of getting something done on things like climate, election protection, and so on...Democrats will wind up empty handed because they couldn't bring themselves to take some easy wins. I think Ruy's got a good case here and encourage you to read it, and his main thesis is summed up in this paragraph:
The thread that runs through all these failures is the Democratic Left’s adamant refusal to base its political approach on the actually-existing opinions and values of actually-existing American voters. Instead they entertain fantasies about kindling a prairie fire of progressive turnout with their approach, despite falling short again and again in the real world. It hasn’t worked and it won’t work.
Last but not least, from Nicole D. in North Carolina:
Can I start The Handmaid's Tale, or is it #toosoon?
I am guessing this may be a reference to the leaked Supreme Court draft around Roe, to which I would suggest waiting for the actual ruling to come out.
But to take the question at face value, I very much enjoyed reading The Handmaid's Tale in high school and recommend it as a powerful read. (I have not watched the TV show, so I have no recommendation there one way or another.)
I often confess that I think I took away a lesson from the novel that Margaret Atwood probably did not intend (namely, some other country should invade to liberate these women). This led me, in the early 2000s, to be much more supportive of the deployment of U.S. military force in places like Afghanistan. Again, probably not what Margaret Atwood was going for.
Earlier this week, the Taliban re-instituted the requirement that women wear burqas. And the BBC's Yalda Hakim is unparalleled in her coverage of the way that women have been barred from education since the Taliban took back charge.
So when people say "The Handmaid's Tale is too real!!!" my response is...well, yes, in Afghanistan, very much so.
***
Thanks again for being a reader of Codebook. Do you have a question for the next "Ask Away!" feature? Hop to the comments to start the discussion.
If you enjoyed this post, please share Codebook with a friend or colleague!