A few days ago, I took our golden retriever Casper to a local park for a long run. It was late at night – or, rather, early in the morning – so we had the park virtually to ourselves. I used a Chuckit! to fling an oversized tennis ball against a high chain link fence a few dozen yards from where I stood; this photograph from a different visit gives you a sense of the distance. Casper dutifully raced after the ball and brought it back to me.
After 10 or 15 tosses, I decided to fling the ball diagonally to my left. Rather than race to the new spot, though, Casper darted to where the ball had always previously landed. He then spent five minutes circling around the park trying to find the ball; for a golden retriever he is quite nose-blind. He even went behind me at one point, stopping to sniff the empty Chuckit!. At that point, I mimicked a throw to where the ball was. But he just sped back to the prior landing spot. Finally, I walked to where the ball had landed. Casper followed me and immediately found the ball; what he lacks in sense of smell he makes up in vision.
Curious, I again hurled the ball diagonally to my left. And, once again, Casper darted to where I had thrown the ball the first 10 or 15 times, repeating the same process as before until I walked to the actual landing spot. To his credit, from that point on, he carefully followed the trajectory of the ball, not simply going where he expected – based upon recent history – the ball would be.
I think this is a good analogy for why many political prognosticators forecast the 2022 midterm elections so poorly, while a few of us did very well. They based their conclusions on what had happened in previous first midterms for a newly-elected president, not on what was actually unfolding in the elections: running not to where the ball landed, but where recent history suggested it would land.
In fact, they began running to that spot in November 2021, following poorer-than-expected performances by Democrats in governor’s races in New Jersey and Virginia; election data from Dave Leip’s indispensable Atlas of U.S. Presidential Elections. In the former, Democratic incumbent Phil Murphy won by 3.2 percentage points (“points”), down sharply from his 14.1-point victory four years earlier. In the latter, former Democratic governor Terry McAuliffe lost to Republican Glenn Youngkin by 2.0 points, down sharply from Democrat Ralph Northam’s 8.9-point victory four years earlier. Both elections saw Republican gain 10.9 points from the previous election. Moreover, these were shifts of 12.7 and 12.2 points Republican, respectively, relative to Democratic presidential nominee Joe Biden’s vote share in 2020.
Professional political data analysts like Dave Wasserman of the Cook Political Report did some math, looking ahead to 2022. If Republicans shifted even 10 points from 2020, they reasoned, when Democrats won the total vote cast for the United States House of Representatives (“House”) by 3.1 points, that would give Republicans a landslide-level 6.9-point win on the 2022 House vote. According to my model of House vote/House seats, that equates to a Republican gain of 32.8 House seats, far more than the five they needed to retake the House majority. And that was on the low end of the shift – the upper end (12.2 to 12.7 points) equated to a Republican gain of roughly 40 House seats. Republican leaders did similar math, with House Minority Leader Kevin McCarthy (R-CA) confidently predicting a gain of up to 60 seats.
To be fair, three of the previous four first-presidential-midterm elections had been similar “shellackings,” as then-President Barack Obama termed Democrats losing 63 House seats in 2010. In 1994, Democratic president Bill Clinton watched his party lose 54 House seats, ceding control of the majority for the first time in 40 years. More recently, Republicans under President Donald Trump lost 41 House seats – and the majority – in 2018. Throw in Biden’s relatively low approval rating, which has hovered between 37.9 and 43.9% for just over a year, and such predictions did not seem outrageous.
Except…even in November 2021, prognosticators were not following where the ball truly was going. After plummeting to a low of R+4.9 in mid-November 2021, based upon a 10-poll rolling weighted average, the generic ballot – whether voters prefer a generic Democrat or a generic Republican in their district – rose sharply, then averaged R+1.4 through the end of May 2022. If these polls were accurate, Democrats would still likely lose their House majority, but by closer to 15 seats, not at least double that number.
Then, on June 24, 2022, the political landscape fundamentally changed: the United States Supreme Court ruled in Dobbs v Jackson that anti-abortion laws were, in fact, constitutional, upending nearly 50 years of precedent. For the first time, the Supreme Court had stripped Americans of a right it had itself granted to them. I analyzed generic ballot polls conducted by the same pollster (n=20) before and after June 24, finding a mean 1.8-point swing towards Democrats. On August 2, Kansas voters defeated an anti-abortion constitutional amendment by 18 points, after polls had shown the two sides essentially tied. Three weeks later, Democrat Pat Ryan won a special House election in New York by 2.3 points despite trailing in a handful of public polls by six or seven points; Ryan focused sharply on abortion rights in his campaign. Just one week later, Mary Peltola became the first Democrat to win Alaska’s at-large House seat in nearly 50 years, again outperforming publicly-available polling.
There were other signs of Democratic strength as well, most notably a post-Dobbs surge in voter registrations that leaned heavily young, female and Democratic. But…the “Republicans will dominate the midterms” narrative had already taken hold, and the same articles that touted these registration gains also noted strong Republican edges on prior registrations as well as concerns about inflation and high gas prices (which, to be fair, were unusually high in early September 2022).
What is so maddening about these narrative blinders is just how little evidence existed for them. Going back to 1960, there have only been nine midterm elections two years after a president is elected for the first time; I exclude 1966, because Democrat Lyndon Johnson had already ascended to the presidency in November 1963. In the first five of these elections (1962, 1970, 1978, 1982, 1990), the party in the White House lost an average of 12.6 House seats, ranging between two (1962) and 26 (1982); adding the 47 seats won by Republicans in 1966 bumps the average up to 18.3, albeit with a median of 13.5. In the four most recent such elections, though, the average House seat loss is 37.5; exclude the anomalous 2002, when Republicans gained eight seats in the first midterm elections since September 11, 2001 (under Republican president George W. Bush), and the average jumps to an astonishing 52.7 House seats. Overall, the post-1960 average House seat loss by White House party in its first midterm elections was 23.7 going into 2022 (26.0 including 1966).
I describe the 2002 House election results as “anomalous” because it was the only such election since 1960 in which the White House party gained House seats. But was it truly so? Democrats lost only two seats in 1962, while Republicans lost only eight seats in 1990; in both cases, the sitting president – John F. Kennedy and George H. W. Bush – was relatively popular, as was the younger Bush in 2002. Still, in three out of nine recent midterm elections – dating back six decades – the White House party lost fewer than 10 seats. But two of those elections were more than 30 years ago, while the third was a rally-around-the-flag fluke.
But when you only have at most 10 such elections in a 60-year span, it is hard to distinguish the fluke from the norm: there simply are too few data points. Even going back a century to Republican president Warren Harding’s first midterms in 1922 (Democrats gained 77 seats) only adds four such elections – unless you count Calvin Coolidge in 1926 and Harry Truman in 1950, both of whom ascended to the presidency upon the death of the president. At most, then, there are only 16 such elections in the past 100 years.
With so few data points, and with the most recent ones being as high as they are, it was not surprising that recency bias led to confident predictions of a “red wave.” Never mind that even more recent events – Dobbs, August elections and voter registrations – were ignored in favor of data from 1994, 2010 and 2018; clearly confirmation bias was also at work, as was the need to maintain professional reputations and institutional points of view. As a data analyst operating purely independently, I avoided at least the latter forms of bias.
Then came the post-Labor Day polls, and as I have written previously, prognosticators focused even more on where they thought the ball was going to land, not where the events of the summer had suggested it easily could land.
***********
In the first part of this post-mortem, I calculated Democrats overperformed my final polling averages by a mean 1.2 points in the 16 most-polled gubernatorial elections (excluding Florida) and by a mean 2.3 points in the 11 most-polled United States Senate (“Senate”) elections (excluding Florida). This confirmed my experience-based hunch polling averages were underestimating Election Day Democratic margins by 1-3 points. One piece of evidence for my “hunch” was the events of the summer: Democrats and Democratic positions strongly overperformed available polling, while the pool of possible (read: “likely”) voters shifted younger, female and Democratic.
A second piece of evidence, though, was just which polling firms were assessing key Senate and gubernatorial races. Table 1 lists the 57 pollsters to assess one election (generic ballot, Senate, governor) at least three times, with at least two assessments after Labor Day.
Table 1: Number of 2022 generic ballot, Senate election and gubernatorial election polls by selected pollsters
| Pollster | Generic Ballot | Senate | Governor | TOTAL | ||||
| # | #>LD | # | #>LD | # | #>LD | # | #>LD | |
| Trafalgar Group | 12 | 5 | 34 | 23 | 28 | 20 | 74 | 48 |
| Emerson College | 10 | 2 | 28 | 19 | 27 | 19 | 65 | 40 |
| Data For Progress | 10 | 5 | 17 | 15 | 20 | 17 | 47 | 37 |
| Insider Advantage/ Opinion Savvy | 9 | 2 | 17 | 16 | 14 | 13 | 40 | 31 |
| Cygnal | 9 | 3 | 15 | 10 | 15 | 13 | 39 | 26 |
| YouGov | 107 | 16 | 3 | 3 | 14 | 6 | 124 | 25 |
| Morning Consult | 119 | 24 | 119 | 24 | ||||
| Fox News/ Beacon Research/ Shaw & Company Research | 10 | 2 | 12 | 8 | 12 | 8 | 34 | 18 |
| Ipsos | 27 | 14 | 27 | 14 | ||||
| Suffolk University | 10 | 6 | 14 | 8 | 24 | 14 | ||
| SurveyUSA | 8 | 4 | 18 | 9 | 26 | 13 | ||
| OH Predictive Insights | 10 | 7 | 4 | 2 | 14 | 9 | ||
| Amber Integrated | 6 | 6 | 6 | 3 | 12 | 9 | ||
| Center Street PAC | 6 | 5 | 6 | 4 | 12 | 9 | ||
| Ascend Action | 6 | 6 | 3 | 3 | 9 | 9 | ||
| Rasmussen Reports/ Pulse Opinion Research | 31 | 8 | 31 | 8 | ||||
| East Carolina University | 7 | 5 | 3 | 2 | 10 | 7 | ||
| Big Village | 11 | 6 | 11 | 6 | ||||
| Quinnipiac University | 4 | 2 | 7 | 4 | 11 | 6 | ||
| Marquette University Law School | 5 | 3 | 5 | 3 | 10 | 6 | ||
| University of Georgia School of Public and International Affairs | 4 | 3 | 4 | 3 | 8 | 6 | ||
| Saint Anselm College | 4 | 2 | 4 | 2 | 8 | 4 | ||
| Selzer & Co | 3 | 2 | 4 | 2 | 7 | 4 | ||
| Franklin & Marshall College | 3 | 2 | 3 | 2 | 6 | 4 | ||
| Siena College | 3 | 2 | 3 | 2 | 6 | 4 | ||
| University of New Hampshire | 3 | 2 | 3 | 2 | 6 | 4 | ||
| EPIC-MRA | 5 | 4 | 5 | 4 | ||||
| Navigator Research | 5 | 4 | 5 | 4 | ||||
| NBC/Hart Research/ Public Opinion Strategies | 9 | 3 | 9 | 3 | ||||
| Public Policy Polling | 6 | 3 | 6 | 3 | ||||
| Change Research | 5 | 3 | 5 | 3 | ||||
| The Political Matrix/ The Listener Group | 5 | 3 | 5 | 3 | ||||
| Moore Information | 4 | 3 | 4 | 3 | ||||
| WPA Intelligence | 3 | 3 | 3 | 3 | ||||
| Echelon Insights | 24 | 2 | 24 | 2 | ||||
| McLaughlin & Associates | 17 | 2 | 17 | 2 | ||||
| Harris Insights & Analytics | 11 | 2 | 11 | 2 | ||||
| Marist College | 9 | 2 | 9 | 2 | ||||
| NewsNation | 9 | 2 | 9 | 2 | ||||
| SSRS | 9 | 2 | 9 | 2 | ||||
| The Winston Group | 8 | 2 | 8 | 2 | ||||
| Monmouth University | 7 | 2 | 7 | 2 | ||||
| University of Texas at Tyler | 7 | 2 | 7 | 2 | ||||
| ABC News/ The Washington Post | 5 | 2 | 5 | 2 | ||||
| Dan Jones & Associates | 5 | 2 | 5 | 2 | ||||
| Glengariff Group | 5 | 2 | 5 | 2 | ||||
| GQR Research | 5 | 2 | 5 | 2 | ||||
| Susquehanna Research & Polling Inc. | 5 | 2 | 5 | 2 | ||||
| IBD/TIPP | 4 | 2 | 4 | 2 | ||||
| Clout Research | 3 | 2 | 3 | 2 | ||||
| co/efficient | 3 | 2 | 3 | 2 | ||||
| Data Orbital | 3 | 2 | 3 | 2 | ||||
| Nelson Research | 3 | 2 | 3 | 2 | ||||
| OnMessage | 3 | 2 | 3 | 2 | ||||
| Siena College/ The New York Times Upshot | 3 | 2 | 3 | 2 | ||||
| Sooner Poll | 3 | 2 | 3 | 2 | ||||
| University of California, Berkeley | 3 | 2 | 3 | 2 | ||||
| TOTAL | 485 | 120 | 234 | 167 | 257 | 170 | 976 | 457 |
| % ALL POLLS | 81.1% | 87.6% | 38.4% | 58.6% | 39.4% | 56.3% | 52.6% | 63.0% |
The 57 pollsters in Table 1 account for just over half of all 2022 generic ballot, Senate election and gubernatorial election polls – and nearly two-thirds of the post-Labor-Day assessments of these elections. Notably, though, generic ballot polls are far better represented here than Senate/gubernatorial elections. The 26 generic ballot pollsters in Table 1 account for 81.1% of all generic ballot polls conducted since January 1, 2001 and 87.6% of those conducted after Labor Day. By contrast, the 28 Senate election pollsters and 32 gubernatorial election pollsters in Table 1 account for just under 40% of all such polls conducted since November 1, 2001 and a small majority of those conducted since Labor Day.
Of the 485 generic polls summarized in Table 1, 308 (63.5%) were conducted by just five pollsters: Morning Consult, YouGov, Rasmussen Reports/Pulse Opinion Research, Ipsos (who polled all adults only) and Echelon Insights. By contrast, the six pollsters who conducted at least 15 Senate or gubernatorial polls – Trafalgar Group, Emerson College, Data For Progress, Insider Advantage/Opinion Savvy, Cygnal and SurveyUSA – account for only 241 polls (49.1%) of the 491 Senate and gubernatorial elections summarized in Table 1. Setting aside the 226 combined generic ballot polls conducted by Morning Consult and You Gov (who assessed PA Senate, PA governor and TX governor a combined 17 times), the biggest “players” in the 2022 midterm elections were Trafalgar (n=74), Emerson (65), Data For Progress (47), Insider Advantage (40), Cygnal (39) and the polls conducted for Fox News by Beacon Research/Shaw & Company Research (34); these six pollsters and YouGov were the only pollsters to assess elections in all three categories.
While this small group of pollsters dominated the 2022 midterm elections, familiar names like Change Research (Iowa Senate election only) and Public Policy Polling (Washington Senate election only) mostly stayed on the sidelines. Highly-rated pollsters like ABC News/The Washington Post, Marist College, Monmouth University and NBC News/Hart Research/Public Opinion Strategies, Siena College/The New York Times also assessed the generic ballot almost exclusively. On the flip side, while highly-rated Suffolk University and Quinnipiac University conducted a total of 35 Senate and gubernatorial polls, each conducted only one post-Labor-Day generic ballot poll. The 13 Quinnipiac University and 6 Suffolk University generic ballot polls overstated Democratic strength by only 0.6 and 0.2 points, respectively, somewhat improving their overall performance (Table 2).
So, how did they perform?
Table 2 shows the average deviation from the final election result for the 41 pollsters who assessed the generic ballot and/or at least two Senate/gubernatorial elections; negative values indicate pollsters overestimated Democratic strength. To calculate average deviation is used a time-weighted average of the raw margins (i.e., unadjusted for historic bias) Pollster rating and historic bias (positive values indicate Republican overestimation) come from FiveThirtyEight pollster ratings; pollsters with a C+ rating and no bias have insufficient data for rating assignation. As of this writing, Republicans lead on the total vote cast for House by 2.9 points.
Table 2: Average 2022 polling miss, pollsters assessing generic ballot and/or >1 statewide elections
| Pollster | Rating | Bias | # Elections | Generic | Senate | Governor | MEAN |
| Saint Anselm College | A/B | -0.5 | 2 | 7.1 | 2.4 | 4.8 | |
| Trafalgar Group | A- | 1.3 | 19 | 3.6 | 5.1 | 4.8 | 4.5 |
| Insider Advantage/ Opinion Savvy | B | -1.1 | 8 | 1.6 | 3.5 | 3.6 | 2.9 |
| Echelon Insights | B/C | -0.1 | 1 | 2.8 | 2.8 | ||
| Rasmussen Reports/Pulse Opinion Research | B | 1.5 | 1 | 2.4 | 2.4 | ||
| University of New Hampshire | B- | -2.4 | 2 | 5.4 | -1.1 | 2.2 | |
| Data For Progress | B | -3.3 | 12 | 0.5 | 3.3 | 0.6 | 1.5 |
| Marquette University Law School | A/B | -0.4 | 2 | 0.8 | 1.7 | 1.3 | |
| McLaughlin & Associates | C/D | 1.6 | 1 | 0.9 | 0.9 | ||
| Monmouth University | A | -2.1 | 1 | 0.5 | 0.5 | ||
| Fox News/Beacon Research/Shaw & Company Research | A | -1.8 | 9 | 0.1 | 0.3 | 0.9 | 0.4 |
| East Carolina University | B/C | -1.6 | 3 | 1.0 | -0.2 | 0.4 | |
| Emerson College | A- | -0.8 | 17 | 0.6 | 1.0 | -0.7 | 0.3 |
| University of Georgia School of Public and International Affairs | B/C | -0.5 | 2 | 0.1 | 0.4 | 0.3 | |
| IBD/TIPP | A+ | 0.8 | 1 | -0.4 | -0.4 | ||
| Cygnal | B+ | -2.1 | 7 | -1.2 | -1.1 | 0.5 | -0.6 |
| ABC News/The Washington Post | A+ | -0.9 | 1 | -0.7 | -0.7 | ||
| Harris Insights & Analytics | B+ | 1.1 | 1 | -0.9 | -0.9 | ||
| Susquehanna Research & Polling Inc. | B+ | -0.6 | 1 | -1.0 | -1.0 | ||
| YouGov | B+ | -0.7 | 4 | -5.4 | 0.9 | 1.0 | -1.2 |
| Franklin & Marshall College | B/C | -0.5 | 2 | -0.4 | -2.1 | -1.3 | |
| Suffolk University | B+ | -0.9 | 7 | -1.9 | -0.8 | -1.4 | |
| OH Predictive Insights | B/C | 0.7 | 4 | 0.2 | -3.4 | -1.6 | |
| Siena College/The New York Times Upshot | A+ | -1.1 | 1 | -1.9 | -1.9 | ||
| SSRS | C | -2.7 | 1 | -1.9 | -1.9 | ||
| NewsNation | C+ | 1 | -2.2 | -2.2 | |||
| NBC/Hart Research/Public Opinion Strategies | B+ | 1.5 | 1 | -2.5 | -2.5 | ||
| Selzer & Co | A+ | -0.1 | 2 | -3.4 | -1.8 | -2.6 | |
| Big Village | B | 0.0 | 1 | -3.4 | -3.4 | ||
| SurveyUSA | A | 0.0 | 5 | -5.4 | -2.0 | -3.7 | |
| GQR Research | B | -2.1 | 1 | -3.8 | -3.8 | ||
| The Winston Group | A/B | -1.1 | 1 | -4.3 | -4.3 | ||
| Ipsos | B- | -1.6 | 1 | -4.7 | -4.7 | ||
| Marist College | A | -0.1 | 1 | -4.8 | -4.8 | ||
| Quinnipiac University | A- | -0.5 | 3 | -5.9 | -4.2 | -3.2 | |
| Morning Consult | B | -2.9 | 1 | -5.5 | -5.5 | ||
| Siena College | A | 0.5 | 2 | -6.3 | -8.2 | -7.3 | |
| Navigator Research | B/C | 0.5 | 1 | -7.6 | -7.6 | ||
| Center Street PAC | C+ | 6 | -11.7 | -9.7 | -10.7 | ||
| Amber Integrated | B/C | 0.0 | 3 | -15.8 | -11.1 | -13.5 | |
| Ascend Action | C+ | 3 | -14.0 | -14.8 | -14.4 | ||
| Average | B+/B | -0.6 | 3.5 | -1.5 | -1.7 | -2.0 | -2.0 |
These pollsters generally fared well, overestimating Democratic strength on the generic ballot by only 1.5 points and in Senate and gubernatorial elections by 1.7 and 2.0 points, respectively; across the board, Democratic strength was overestimated by only 2.0 points. These 41 pollsters were respectable, averaging a B+/B pollster rating, with a B+ median. Historically, these pollsters would have been expected to overestimate Democratic strength by 0.6 points, 1.4 points lower than the final-election-margin average.
That said, three of the six most active pollsters – Trafalgar (R+4.8), Insider Advantage (R+2.9) and Data For Progress (R+1.5) – consistently showed Democrats performing far worse than their final Election Day margins. Like other pollsters, meanwhile, Saint Anselm College and the University of New Hampshire underestimated Democratic Senator Maggie Hassan’s final 9.1-point-margin by an average of 6.3 points! Marquette University Law School’s polls, however, only somewhat underestimated the strength of Democratic Senate nominee Mandela Barnes (0.8) and Democratic governor Tony Evers (1.7).
At the other end are some newer pollsters – Center Street PAC (D+10.7), Amber Integrated (D+13.5), Ascend Action (D+14.4) – who dramatically overestimated Democratic strength in Senate and gubernatorial elections, the latter two in the three Oklahoma elections (Senate, Senate special election, governor). Like other pollsters, Siena College overestimated support for two New York Democrats, Senator Chuck Schumer and Governor Kathy Hochul, by a mean 7.3 points. Quinnipiac once again significantly overestimated Democratic strength, albeit in only three elections, this time by a mean 5.1 points. More surprising is the Democratic overestimation of 2.6 and 3.7 points, respectively, by two highly-rated pollsters: Selzer & Co. (IA Senate, IA governor) and SurveyUSA (GA Senate, WA Senate, GA governor, MN governor, NM governor). Finally, 11 pollsters overestimated Democratic strength on the generic ballot by at least 1.9 points: Siena College/NYT, SSRS, NewsNation, NBC/Hart/POS, Big Village, GQR, The Winston Group, Ipsos, Marist College, Morning Consult and Navigator Research.
That leaves 14 pollsters who, on average, performed well in the 2022 midterm elections, six of whom primarily assessed the generic ballot: McLaughlin & Associates, Monmouth, IBD/TIPP, ABC/WP, Harris Insights & Analytics and Susquehanna Research & Polling Inc. Three others focused exclusively on individual states like Georgia, North Carolina and Pennsylvania. Finally, five regularly assessed the generic ballot AND ≥3 Senate and gubernatorial elections: YouGov (D+1.2, GB+3), Suffolk University (D+0.8, GB+7), Cygnal (D+0.6, GB+6), Fox News (R+0.4; GB+8) and the reigning accuracy champion, Emerson College (R+0.3, GB+16); Table 3 details Emerson College’s 2022 performance.
Table 3: Deviance of 2022 Emerson College polls from Election Day margins
| Election | Deviation | |
| Arithmetic | Absolute Value | |
| Generic Ballot | 0.6 | 0.6 |
| Connecticut Senate | 2.4 | 2.4 |
| Georgia Senate | -0.3 | 0.3 |
| Nevada Senate | 3.6 | 3.6 |
| New Hampshire Senate | 3.6 | 3.6 |
| New York Senate | -4.5 | 4.5 |
| North Carolina Senate | 1.1 | 1.1 |
| Ohio Senate | -1.9 | 1.9 |
| Pennsylvania Senate | 4.1 | 4.1 |
| Connecticut Governor | 2.0 | 2.0 |
| Georgia Governor | -1.8 | 1.8 |
| Nevada Governor | 0.3 | 0.3 |
| New Hampshire Governor | 1.1 | 1.1 |
| New York Governor | -3.8 | 3.8 |
| Ohio Governor | -7.6 | 7.6 |
| Pennsylvania Governor | 5.8 | 5.8 |
| Texas Governor | -2.0 | 2.0 |
| Mean | 0.2 | 2.7 |
| SD | 3.4 | 2.0 |
On average across these 17 contests, Emerson College almost perfectly matched the final Democratic-minus-Republican margin, deviating from the final margin (regardless of direction) by a mean 2.7 points. The pollster’s worst performances came in elections other pollsters missed badly as well, including gubernatorial elections in New York, Ohio and Pennsylvania and Senate elections in New York and Pennsylvania.
But while Emerson College gets the 2022 election accuracy crown by virtue of assessing 17 different contests at least three times each, it is almost a coin flip with Fox News. On average across nine contests, Fox News had an identical average arithmetic miss of 0.2, with a lower standard deviation of 1.9, and the corresponding either-direction values are 1.6 and 1.0; the largest deviation was underestimating Evers’ Election Day margin by 3.5 points.
Finally, I observe that while there is no relationship between historic bias and mean deviation (r=-0.03), there is a suggestion that the higher the pollster rating (0.19) and the more elections a pollster assessed (0.25), the more Democratic strength was underestimated. Curiously, the association with pollster rating reverses (-0.25) when the absolute value of the mean deviation is used: the better the pollster, the smaller the deviation in either direction; the five best-performing pollsters had an average rating of A-/B+. Using the absolute value of the mean deviation removes the association with number of elections (-0.06), while there is a slight hint that the more Republican the historic bias, the worse the pollster performed (0.15). These data lend credence to the idea that highly-rated Republican-leaning pollsters, most notably Trafalgar, “flooded the zone” with overly optimistic Republican polls.
Still, what I find most reassuring is that eight pollsters overestimated Republican strength (mean deviation 2.8), 14 were within a point or so (-0.3), and 19 overestimated Democratic strength (-5.4). While still a pro-Republican imbalance, the average deviations are far more evenly distributed than in recent elections. Now, all pollsters need to do is better estimate Republican strength in the most Republican states.
Until next time….and if you like what you read on this website, please consider making a donation. Thank you.
Just Bear With Me... 
One thought on “2022 Elections: A Post-Mortem (Part 2)”