Ask Away! Kristen answers your January mailbag questions.

Yes, I am taking questions about Wally. In fact, I prefer them.

Welcome to Codebook, a newsletter that decodes our world through polling and research. This edition—"Ask Away!"—will be a new monthly feature, rounding up the best reader questions of the last month.

Please subscribe here and follow my page on Facebook for the latest. And don't forget: Every Monday is a premium subscriber-only roundup of the numbers you need heading into the week.

I love a good "mailbag." I love them when Andy Greenwald and Chris Ryan do them on "The Watch" podcast. I love them when fantasy football writers do them. I'm very excited for Josh Barro's "Mayonnaise Clinic" mailbag moving forward.

And thus..."Ask Away!"

Do you have a question about polling? (Many of you seem to!) Working in politics or media? What are the best chili peppers to grow in your garden? Can we just talk about Wally for the next thousand words of this newsletter? (Why not!)

This is where I'll answer your questions. If you want to get in for next month, there are a few ways to do it.

1. Comment here, on Bulletin, under any post. I see all the comments!

2. Comment on a post on my Kristen Soltis Anderson writer Facebook page or send a message to that profile. I read your notes!

3. Reply to me on other social media platforms (Instagram and Twitter: @ksoltisanderson).

4. Send a carrier pigeon or a bouquet of flowers.

On to your questions!

***

First question comes from Jack D., who I will henceforth refer to as Publix Jack due to his bio.

I’ve never been polled—why? How do they come up with ‘who’ is polled? Why do we hear most polls ‘over-poll’ Democrats?

Good questions, Publix Jack. I hear these questions a lot, so I figured it would be best to tackle them right off the bat. First, most people will probably not get polled for a political poll during their lifetimes (or, a pollster will call, they'll screen it thinking it’s spam, and they'll go on thinking they've still been left un-sampled).

The reality is that you are probably polled a lot, but most polling isn't about politics. If you've ever been asked for your feedback on a consumer experience, your thoughts on a product or service, etc....that's opinion research. Political polling is a teeny-tiny piece of a much broader "opinion research" ecosystem that has a lot more resources going to study what the next pharmaceutical drug should be named or what flavor of potato chip is going to be the next big hit. (Hope it's spicy.)

But! Think about it this way: There are 168 million registered voters in the United States as of the 2020 election. In any given year, a major news organization will probably do a poll or two a month, probably with a sample size of one thousand or so. Now, they'll call a lot more than a thousand people to get those completed interviews done, but remember my first point—you might have been one of those people and you screened the call assuming it was spam.

And though they're not the only ones doing political polling -- for every public media poll, there are probably dozens of private polls being done for campaigns here and there -- the sum total of every interview done for a political poll in America in a given election cycle is still just an itsy bitsy slice of the full electorate.

It it's likely you're right—you, Publix Jack, weren't called. The odds of you being called are very low. But! Because math is wild (shout out the central limit theorem), if you take a sufficiently large, truly random sample of people out of an absolutely enormous pool of possible respondents, the odds are shockingly very good that if you do that enough times you'll get a pretty good picture of reality. Each individual survey has a chance of being super off-the-mark, of course, even if you do everything "right," which is why polling averages are valuable.

Trust me—it's not a conspiracy to leave you out of polling samples. It's the cruel math of how big the pool of adults or registered voters is compared to the number who actually get picked up in the random sampling process. If you do get called, please take the poll!

Now, for who gets polled? There's a level of randomness involved in any decent poll, but a pollster must also choose what population they want to study. Sometimes, you just want "adults" because you care about the views of anyone who is old enough to be surveyed. Sometimes, you just want "registered voters" because you're focused on election-related issues and that's the more relevant group. Or, as elections near, pollsters start looking at "likely voters," a somewhat arbitrary designation meaning that the pollster believes you are likely to turn out to vote. That's a whole other issue for a later time.

But it feeds into the third question, which is: Why do polls often include more Democrats than Republicans? (A few of you asked this!) I'll assume you're talking about national polls. The reality is that there isn't an exactly even number of Americans across the country who identify one way or the other. Gallup has tracked this for a while and found that up until just this past month, Democrats outnumbered Republicans for most of the last few decades. Republicans pulling into the lead on party identification was a big deal!

It's the same reason you wouldn't want the same number of men and women in your poll sample; there aren't the same number of men and women out there. But not everyone turns out to vote, either. You don't want exact balance; you want representativeness.

Next question!

From reader and Internet friend "Password Is Taco":

When is issue polling informative, or is it always a scam, designed to get the answer the questioner wanted?

I do a lot of issue polling in my work, so even though many of my dearest friends generally hate it and find it useless, I will defend it—on certain grounds.

First thing’s first: There's a LOT of issue polling you, the viewer at home, will never see. The vast majority of issue polling I do is for private clients who have a genuine interest in knowing: What do Americans think about issue X? Maybe it's because they want to know if people are ready to come with pitchforks and burn down their HQ; maybe it’s because they are a foundation making a lot of grants in a certain arena and want to know if Americans think it’s an issue that needs fixing; maybe they're a financial firm trying to gauge what legislation Congress will feel pressure to tackle (so they can go calculate if and how it will affect their positions in the market).

The bottom line is that in all of those cases, it is really useful to do rigorous research that asks people for their views and does so in a few different ways so that something like question structure isn't inadvertently giving you a false read. I'm not interested in research that will give me the wrong impression of public opinion, because then I won't be serving my client very well!

But most issue polling you see is publicly released for one of three reasons:

1. It is an organization that is in the business of contributing to our broader body of knowledge (think Pew Research Center, PRRI, etc.)—in which case their motives are probably pretty pure, and while that doesn't mean their research is infallible, it's being done in good faith.

2. It is an organization that knows people are curious about public opinion, and it wants to meet that need. A lot of my work falls into this category (as well as the former). My firm releases data each month because we know it will be of interest, get shared, and possibly raise the profile of our firm as contributing something valuable to our understanding of public opinion. Some of our clients commission polls because they know they will have a lot of interested stakeholders who want data. (This report I worked on for the Konrad Adenauer Foundation is a good example.)

3. It is an organization trying to advance an agenda and prove that its message or position is popular. (Looking at you, Data for Progress.) In this case, I say “buyer beware.” You don't know if there's a poll question they omitted because the result didn't look good for them. You know they probably wrote questions that would pass a credibility "smell test," but put their pinky finger on the scale to get a happy-talk result. You don't have to put these polls in the garbage, but always ask yourself Why is this person releasing this poll?, and let that guide you.

I dug into this on the latest episode of Josh Barro's podcast, so you can listen there for more, but I think the key is to understand the motive behind the research and to see if there are a variety of different ways an issue can be asked about.

Next question comes from Daniel K. on Facebook...

I am an Independent who voted for Biden and who has been disappointed. I thought his press conference yesterday was a disaster... Is Biden's approval rating related to the feelings of most Americans, me included, that he is not all there? Is Biden just a really bad public speaker, or is this a mental acuity issue? What does America think?

This question was raised directly to President Biden last week by a reporter, and there is a little bit of data on this. The most recent poll on it is the POLITICO/Morning Consult poll, which shows by a 40–49 margin, more Americans disagree that "Joe Biden is mentally fit." There's also the NBC poll that I referenced in my Monday premium-subscriber-only post where Biden's numbers have fallen on a lot of characteristics and are pretty poor (only 33%) on whether he has "the necessary mental and physical health to be President."

It does seem to be a concern many Americans have, and even some who voted for him or approve of the job he is presently doing don't necessarily think he should run again. This mid-December Yahoo/YouGov poll asked about it, and only 22% of Americans said they wanted to see Biden run for President again. Not really a great place for him to be.

Last question! From Noah Lucas of Kentucky...

Can we just ask about Wally?

Always. For readers who are unfamiliar, Wally is my elderly golden retriever. He's an immigrant from Turkey and has been in my household for the last three years. He's an expert thief, he loves carbs and belly rubs, and every day I post a picture of him for my #dailywally (a dose of joy on the otherwise dark and scary Internet).

In February's "Ask Away!", Wally will also be fielding your questions.

Thanks to everyone who submitted, I'm sorry I couldn't get to all of them and hope to tackle more in coming weeks! And if you missed the boat this time, send your questions along and I'll hang on to them for next month's edition.

***

Thanks again for being a reader of Codebook. Do you have a question for next month's "Ask Away!" feature? Hop to the comments to start the discussion.

If you enjoyed this post, please share Codebook with a friend or colleague!

Come say hello on: Facebook, Twitter, Instagram

(Cover photo: E2 Summit)