New FiveThirtyEight Rank: Echelon Is Top 5 for 2021-2022 Election Cycle
The lesson here is to always hire a team of people smarter than you are.
The polls are not broken. I said as much just a few days after the elections, when Lachlan Markay at Axios reached out to see how I felt about things.
Kristen Soltis Anderson, a founding partner at Republican polling firm Echelon Insights, told Axios: "I completely reject the idea that this was a polling miss."
"If you were mostly following the 'blue-chip' pollsters, and — crucially — if you weren't mentally adding in a 'shy Trump' buffer to those results, you should have expected a lot of what we are seeing."
I also talked to Derek Thompson at The Atlantic, whose write-up has the absolutely perfect sub-head: “Overall, the polls were basically right. It was the vibes that were wrong.”
As more and more returns came in and the picture became clearer, I believe this view holds up well. I get asked a lot, wellllllllll but the polls are broken, right? Even the President of the United States is on this beat these days. But after 2022, I have felt much better about saying, no, my industry is still doing the work to get it right. (Whether consultants speaking to reporters on background got it right is a separate question.)
If you want to go further down this rabbit hole, here’s Isaac Chotiner at The New Yorker who discussed things with The New York Times’ Nate Cohn. There’s Nate Silver’s write up of how FiveThirtyEight’s forecasts did, which were fueled by large volumes of polling data. Or here’s FiveThirtyEight’s assessment of how individual pollsters did this past cycle.
Which brings me to the real point of today’s newsletter, which is to brag on my team at Echelon Insights.
The new FiveThirtyEight pollster ratings are out, and we are top five for the cycle in terms of average error - and unless I’m missing something about the four above us on the list, I believe that makes us the pollster with a partisan affiliation (when we poll for political clients, we poll on the Republican side) highest up on the list.
Of all the polls evaluated this cycle, Echelon was rated as top five in terms of “average error” of all polls FiveThirtyEight evaluated with fresh polling in the 2021-2022 election cycle. You may recall that we got the Virginia governor’s race margin right within a point in 2021, and this cycle we had a number of polls we produced that were pretty close to the mark. You can take a look at them here.
Prior to this ratings update, Echelon had been rated B/C, which is a perfectly fine rating and is considered “provisional” because we produce fewer public-facing polls than the 20 or so recent polls that would be needed to get a firmer rating. We have now been upgraded to A/B, which is the best rating you can get at the number of public polls we have released. (A great deal of our work is private and never gets released, and thus doesn’t affect this one way or the other.)
I am writing this newsletter to celebrate, because it is really nice to see the team get honored for this. Since starting Echelon around nine years ago, we have put together an incredibly bright group of analysts and data scientists who have taken our initial vision - let’s do polling that benefits from all the other data sources we can get our hands on - and have made it shine in practice.
I know that no matter how hard we work, there will be elections in the future where the numbers just break funny and it’ll time to go back to the drawing board. We always have to keep learning. But I also think it is important to celebrate the Ws when they come your way. So to my team - THANK YOU. You all are the very best and it is an honor to work with you every day.