About those recent Emerson College polls…

I first wrote about Emerson College polls here, using the fact their polls of hypothetical 2020 matchups between a Democrat and President Donald J. Trump force respondents to choose a candidate (i.e., have 0% “other/undecided”) to assess Harris X polls, which often have very high proportions “other/undecided.” At the time, I concluded “other/undecided” Harris X polls respondents likely would vote for the 2020 Democrat presidential nominee roughly 2-1.

[Eds. note: I have since updated this post here and here]

When I wrote that post in June, margins reported by Emerson College polls were broadly in line with those reported by other, non-Harris-X polls of these hypothetical matchups. Since the end of August, however, they have taken a sharply Republican turn compared both to previous Emerson College polls and to all other polls, based upon analyses of my WAPA (weighted-adjusted polling average).

Table 1 clearly demonstrates this pro-Republican shift.

Table 1: Polling margin for specified 2020 Democratic presidential nominees over President Trump, Emerson College vs. All Other Polls and January-August 2019 vs. September-November 2019

Biden–National
	Emerson	All Other
Jan-Aug	7.24	8.35
Sep-Nov	0.29	8.03	Difference
	-6.95	-0.32	-6.63
Warren–National
	Emerson	All Other
Jan-Aug	1.52	1.82
Sep-Nov	1.29	4.66	Difference
	-0.23	2.84	-3.07
Sanders–National
	Emerson	All Other
Jan-Aug	4.28	4.99
Sep-Nov	0.12	5.10	Difference
	-4.16	0.11	-4.27

While the post-August-2019 Republican lean of Emerson College polls for the three current polling leaders for the 2020 Democratic presidential nomination—former Vice President Joe Biden, Massachusetts United States Senator (“Senator”) Elizabeth Warren and Vermont Senator Bernie Sanders—is apparent, it is most notable for Biden. Thus, while in all other polls, Biden’s WAPA declined modestly from 8.35 to 8.03, in the Emerson College polls it dropped from 7.24 to essentially even! Put another way, there was a 6.63 shift Republican in Emerson College polls for Biden starting around September 1, 2019. The shift was similar for Sanders—4.16 more Republican in Emerson College polls and no meaningful shift in other polls. Warren, meanwhile, had a sharp increase in her WAPA—from 1.82 to 4.66—in non-Emerson-College polls after September 1, 2019, while dropping slightly in the Emerson College polls. Overall, across all three candidates, there was a 4.66 pro-Republican shift in Emerson College polls after September 1, 2019.[1]

**********

Before addressing what could have caused this shift, let us address what did NOT cause this shift. On November 5, 2019, FiveThirtyEight.com updated its pollster ratings, which I use in calculating WAPA. Emerson College was upgraded from B+ to A-, making it one of the highest-quality pollsters regularly assessing the 2020 presidential election. Moreover, Emerson College’s “mean-reverted bias”—how much more or less Democratic its average polling margins are compared to other pollsters in the same (already-concluded) election—barely changed, shifting from D+0.1 to R+0.0 (which I code as D-0.025). Finally, mean sample size of Emerson College polls—still of registered voters—dropped only slightly after September 1, from 1,120 to 1,043.

But what about other pollsters? If anything, the mix of pollsters assessing the 2020 presidential election improved after September 1, 2019, from B-/B to B, albeit with a slightly more Democratic skew (D+0.2 to D+0.3).

Moreover, here are the averages of unskewed pollster-average margins for Biden, Warren and Sanders across all pollsters with at least a B+ rating,[2] before and after September 1, 2019:

• Biden: 9.3 percentage points (“points”) to 10.8 points
• Warren: 2.4 points to 6.6 points
• Sanders: 5.8 points to 7.5 points

Clearly, both the more recent Emerson College polls and the lower-rated pollsters are finding much closer races between each of these three candidates and Trump than are the higher-rated pollsters. And while I removed the skew from these margins, the higher-rated pollsters have a mean skew of D+0.5—so take these averages with a modicum of salt.

Nonetheless, polls conducted by pollsters similar in quality to Emerson College—albeit with some small percentage of “undecided/other” voters—show the three leading Democratic candidates increasing their hypothesized margins against Trump by 1.5 to 4.2 points after September 1, 2019, while the Emerson College polls show declines in support from 0.2 to 7.0 points.

So what gives?

There are two broad categories of possible, non-mutually-exclusive explanations.

1. Starting September 1, Emerson College pollsters adjusted how they weigh their samples by various demographic factors, such that their polls skew sharply more Republican relative to other high-quality pollsters.
2. Emerson College pollsters, perhaps because they force respondents to make a choice, are capturing a genuine pro-Republican shift in the electorate other high-quality pollsters are missing.

Taking each possible explanation in turn…

Emerson’s sampling methodology did not change after September 1, remaining a combination of Interactive Voice Response (i.e., “robocalls”) of landlines—but not cellphones—and an online voter panel. However, their sample weighting, based upon 2016 turnout, did change subtly—from age, region, income, and education in their August 2019 national polls to age, [interview] mode, party registration, ethnicity and region in their November 2019 national polls.

Given how strongly education is now associated with partisanship, especially among white voters, that could account for at least some of the difference. I am perplexed, however, how interviewing mode is associated with turnout—other than landline-users tending to lean Republican (older, whiter, less urban). For that matter, I am not clear why a poll of registered voters would adjust for turnout at all. Adjust for the relative proportions of these groups in the universe of registered voters, sure—but adjustment for participation rates (i.e., turnout) of various groups in 2016 seems more appropriate for a likely voters model. Still, without seeing the raw data, I will not even speculate how these changes in sample weighting would affect publicly-released polling margins.

There is, meanwhile, an argument to be made that the universe of “decided” voters has drifted Republican in recent months, especially with the announcement by Speaker of the United States House of Representatives (“House”) Nancy Pelosi on September 24, 2019 that the House would formally begin an impeachment inquiry into Trump. While polling at first showed approval for this action higher than disapproval, the difference is closer to even now as House Republicans have rallied behind a president of their party.

Also, while Democrats fared well in recent governor’s races in Kentucky, Louisiana and Mississippi, the final actual margins were an average 2.8 points more Republicans than the final WAPA, perhaps reflecting a substantial Republican bias among “other/undecided” voters—essentially what the Emerson College polls might show. Moreover, the A+-rated Siena College/New York Times Upshot (R+0.3), recently released a set of state-level polls that also show leading Democrats faring less well against Trump than the consensus of other pollsters in those states.

While some combination of these two possible explanations—a change in sample weighting and an actual pro-Republican shift in the electorate—probably accounts for the clear shift towards Trump in recent Emerson College polls, it is entirely possible their last three national polls are showing these shifts purely by chance; even the best sampling strategy will be well wide of the mark at times (this is, in fact, the basic logic behind polling aggregation). However, three such wide sampling “misses” in the same direction and of the same size are extremely unlikely, though far from impossible.

In the end, the best those of us who track election polling can do is throw every publicly-available poll into the analysis, weighting, averaging and adjusting as best we can—all the while remembering that even the best estimates are just that, estimates.

Until next time…please wear a mask as necessary to protect yourself and others – and if you have not already done so, get vaccinated against COVID-19! And if you like what you read on this website, please consider making a donation. Thank you.

[1] Values were similar for South Bend, Indiana Mayor Pete Buttigieg and California Senator Kamala Harris, but with each having only one post-August-2019 Emerson College poll assessing support versus President Trump, I excluded their polls from the analysis.

[2] IBD/TIPP, CNN/SSRS, Fox News, NBC News/Wall Street Journal, ABC News/Washington Post, SurveyUSA, Quinnipiac University. There is no post-August-2019 NBCNWSJ polling for Sanders.