2022 Elections Update: Rethinking polling margins of error

On Labor Day 2022, I published my first set of projections for who will have the majority in the United States House of Representatives (“House”) following the 2022 elections, as well as who will win the 35 elections for the United States Senate and 36 elections for governor. In the ensuing two weeks, an additional 17 generic ballot polls – which I use to estimate the probability Democrats retain their House majority – have been released, along with 49 new Senate polls and 48 new governor’s race polls.

Before reviewing how this polling affects the outlook for November, I explain my new thinking on margins of error. As I noted in the prior essay, I use the total number of persons surveyed within an election to calculate a traditional margin of error (“MOE”) around my weighted-adjusted polling averages, or WAPA. For example, the 13 polls conducted since November 1, 2021 of the Arizona governor’s race between Democratic state Secretary of State Katie Hobbs and Republican television personality Kari Lake surveyed a total of 9,900 people. This equates to MOE=1.0. The current WAPA estimate is 46.7% Hobbs and 44.5% Lake, but the MOE suggests there is a 95% chance the “true estimates” are between 45.7% Hobbs and 45.5% Lake and 47.7% Hobbs and 43.5% Lake; I simply add and subtract 1.0 from each initial estimate.

Through Labor Day, I was multiplying MOE by 2.0 to account for other systematic biases. A week ago, though, I watched a discussion between pollster Cornell Belcher and Chris Hayes on the latter’s MSNBC program All In With Chris Hayes. In this conversation, Belcher made a very important point:

“And until you get the 51 percent, Chris, anything can happen, because these races are not static. If you look at that — you know, that race, say that`s, you know, 44-43, any — that`s a toss-up and anything can happen because, you know what, Chris, we`re spending millions and millions and millions of dollars to in fact, move that rate, to sort of move those numbers around, so you still have a large enough share of voters who are fluid and persuadable to change the outcome of that race.”

Belcher is basically saying that two factors drive how volatile polling for an election is: 1) how close the leading candidate is to 50% and 2) what percentage of respondents choose neither the Democratic or Republican candidate, even after being asked multiple times. Once a candidate tops 50% of the vote in the polling average, that candidate is a near-lock to win. But consider Nevada’s Senate election. Incumbent Democrat Catherine Cortez Masto “only” leads Republican former state Attorney General Adam Laxalt 43.9% to 42.0%, with an unusually high 14.1% choosing neither the Democrat nor the Republican. Based on this information, this election is basically a toss-up, despite MOE on the 1.9 percentage-point margin (“point”) being just 0.9.

Realizing simply multiplying MOE by 2.0 was not sufficient to capture “true” volatility. I calculated three values for each of the 54 Senate and gubernatorial election for which I could find publicly-available polls:

  1. How close polling leader is to 50.1% (i.e., 50.1% – leader%)
  2. Percentage not choosing Democrat or Republican plus 0.2 to account for each candidate reaching 50.1%
  3. Percentage of non-party voters (value #2) polling leader needs to reach 50.1% (i.e., value #1/value #2)

I then created an “MOE volatility multiplier” (“MVM”) by summing value #2/10 and value #3 times 10. Returning to the Arizona governor’s race, Hobbs needs 3.4 additional points to reach 50.1. This is a relatively high 37.5% of the 9.0% not choosing either Hobbs or Lake. Dividing 9.0 by 10 yields 0.9 while multiplying 0.375 by 10 yields 3.75. Summing the two values yields MVM=4.7, which is considerably higher than 2.0.

I set all MVM<0 – eight Senate and six gubernatorial candidates currently over 50% – to 1.0, and I used a threshold of 40.1% for the three-way Oregon gubernatorial election between Democratic former State House Speaker Tina Kotek, Republican former State House Majority Leader Christine Drazan and Independent former state senator (and former Democrat) Betsy Johnson.

***********

The House. As of September 19, 2022, by best estimate is that Republicans lead Democrats on the generic ballot – how a respondent would choose between an unnamed Democrat and Republican if the election in their district were held today – by just 0.3 points. Based upon this value, I estimate the probability Democrats retain a House majority is 22.7%. This is an increase of 2.4 points in two weeks. I also estimate Democrats will lose 11.8 seats, putting them at ~210 (compared to the 222-213 majority they held following the 2020 elections), with a 95% confidence interval of -33.6 to +9.4.

Using only the last 30 polls conducted, though, Democrats lead by 1.1 points, which equates to a 36.9% chance of retaining their House majority and an average loss of 7.3 seats (95% CI: -29.1 to +13.9). For Democrats to be nominal favorites to retain their House majority (p>50%), they need to win nationally by ≥2.3 points.

These probability and seat estimates are well in line with those from FiveThirtyEight.com, whose probabilities range from 28% (Deluxe) to  41% (Classic), with 211 seats assigned >50% probability of being won by the Democrat in the methodologically-conservative Deluxe model. However, if Democrats win every seat with probability>40%, they maintain a bare House majority across all three models.

Bottom line: A narrow House Republican majority is still more likely than not, though Democrats’ position steadily improves.

***********

The Senate. As of September 19, 2022, 211 publicly-available Senate election polls have been conducted since November 1, 2021. These include four polls assessing a final matchup between two Republicans in Alaska: incumbent Senator Lisa Murkowski and former state official Kelly Tshibaka; Murkowski edges Tshibaka 51% to 49%. Only the Senate elections in California, Hawaii, Maryland and Oregon – which incumbent Democrats Alex Padilla, Brian Schatz, Chris Van Hollen and Ron Wyden, respectively, are near-locks to win – and in Alabama, Idaho and North Dakota – which Republicans Katie Britt, Michael Crapo and John Hoeven, respectively, are near-locks to win – do not yet have publicly-available polling.

Table 1 shows updated projections for all 35 Senate elections.

Table 1: Democratic Senate win probabilities and projected final margins, 2022

StateFundamentalsPolling (WAPA)Poll WtFinal Projection
Democratic seatMarginP(D win)MarginP(D win) MarginP(D win)
Hawaii34.2100.0%n/an/an/an/a100.0%
Vermont28.6100.0%14.2100.0%0.8616.3100.0%
Maryland31.4100.0%n/an/an/an/a100.0%
California26.4100.0%n/an/an/an/a100.0%
New York25.4100.0%21.7100.0%0.8622.2100.0%
Connecticut19.199.5%12.999.7%0.8314.099.6%
Washington18.999.5%10.3100.0%0.9011.299.9%
Illinois18.599.4%16.1100.0%0.7116.899.8%
Oregon15.398.0%n/an/an/an/a98.0%
Colorado10.992.3%8.394.7%0.818.894.3%
New Hampshire6.476.3%7.8100.0%0.877.696.9%
Nevada4.771.1%1.964.5%0.812.465.8%
Arizona-4.637.9%7.399.9%0.835.289.1%
Georgia-5.036.5%1.872.8%0.830.666.6%
Estimated Democratic seats lost = 0.9 (range = 0.4 to 1.7)*
 
Republican seat       
Pennsylvania-2.645.1%8.6100.0%0.836.790.7%
Wisconsin-6.032.9%-0.149.3%0.90-0.747.6%
Florida-9.123.2%-4.72.2%0.78-5.66.7%
North Carolina-6.132.7%-0.546.4%0.85-1.344.3%
Ohio-10.120.6%0.252.0%0.79-2.045.2%
Iowa-13.412.9%-7.60.5%0.85-8.52.4%
Alaska-19.44.6%n/an/an/an/a0.0%
South Carolina-19.54.5%-18.50.0%0.74-18.81.2%
Missouri-19.34.7%-12.51.7%0.88-13.42.1%
Indiana-23.22.1%-6.219.3%0.75-10.415.0%
Kansas-24.91.4%-17.40.0%0.73-19.50.4%
Louisiana-25.91.1%-21.40.0%0.74-22.60.3%
Utah-31.20.3%-9.43.4%0.76-14.72.7%
Alabama-29.50.4%n/an/an/an/a0.4%
South Dakota-33.20.1%-32.90.0%0.43-33.10.1%
Kentucky-33.90.1%-15.50.0%0.42-26.10.1%
Arkansas-33.90.1%-12.822.5%0.76-17.917.1%
Idaho-38.40.0%n/an/an/an/a0.0%
North Dakota-39.00.0%n/an/an/an/a0.0%
Oklahoma (1)-41.40.0%-24.20.0%0.73-28.90.0%
Oklahoma (2)-41.40.0%-21.80.0%0.73-27.10.0%
Estimated Republican seats lost = 2.8 (range = 2.0 to 3.9)
Overall expected Democratic seats gained/lost = +1.9 (range = +0.3 to +3.4)

    *Adding or subtracting 3 points to all WAPA, including generic ballot estimates

Italics indicate open seats, while boldface indicates a likely “flip.” “Fundamentals” is the sum of state’s partisan lean (3W-RDM), current generic ballot estimate (-0.3) and incumbency advantage (Senate values: 5.5 points for Democrats, 3.3 points for Republicans); I assign one-third incumbency advantage to Democrats Mark Kelly of Arizona and Raphael Warnock of Georgia because they first won their seats in 2020 special elections. Essentially, this is what one would expect the Democratic margin to be for a generic Democratic vs. a generic Republican. The “fundamentals” probability the Democratic candidate wins is the likelihood of a Democratic margin of ≥0.0000001 given a normal distribution, with mean of +1.3 (mean historic “miss”) and standard deviation of 10.7. Despite improvement on the generic ballot, “on paper,” Kelly and Warnock are modest underdogs, while Democratic incumbents in New Hampshire and Nevada are only modest favorites. On the flip side, the open Republican seat in Pennsylvania – vacated after two terms by Pat Toomey – is essentially a toss-up on paper, albeit with the Republican candidate slightly favored. On average, I now give 23% weight to fundamentals and 77% weight to WAPA, an increase of WAPA weight of 7.0 points in two weeks.

I also modified my categories slightly:

Safe Republican (<5%)

Solid Republican (≥5 to <10%)

Likely Republican (≥10 to <20%)

Lean Republican (≥20 to <30%)

Tilt Republican (≥30 to <40%)

Toss-up (≥40 to ≤60%)

Tilt Democratic (>60 to ≤70%)

Lean Democratic (>70 to ≤80%)

Likely Democratic (>80 to ≤90%)

Solid Democratic (>90 to ≤95%)

Safe Democratic (>95%)

The difference between “Safe” and “Solid” is not always illuminating, so we combine them here:

Safe/Solid Republican (n=13). Incumbents Crapo, Hoeven, John Kennedy (LA), Jim Lankford (OK), Jerry Moran (KS), Rand Paul (KY), Tim Scott (SC) and John Thune (SD) are near-locks to win. Republican nominees Britt, Markwayne Mullin and Eric Schmitt are also prohibitive favorites to win open seats in Alabama, Oklahoma and Missouri, respectively. Republican Senator Mike Lee is heavily favored – for now – to defeat Independent former CIA operative Evan McMullin (effectively the Democratic nominee) in Utah, while I project Murkowski has the edge over Tshibaka.

Likely Republican (n=4). Only one publicly-available poll has been conducted in the Senate races in Arkansas and Indiana, in which incumbent Republicans John Boozman and Todd Young, respectively, should be prohibitive favorites. Still, MVM values of 5.0 and 3.4, respectively, introduce a modicum of uncertainty. Nonetheless, Boozman and Young are projected to win by double-digits.

Meanwhile, incumbents Chuck Grassley in Iowa and Marco Rubio in Florida face strong challenges from retired United States Navy Vice Admiral Michael Franken and House member Val Demings, respectively. Based solely on the fundamentals, Iowa is Likely Republican and Florida is Lean Republican, but polling puts Franken and Demings “only” 7.6 and 4.7 points behind, respectively – albeit Demings closes the gap to 3.7 points using only the 12 polls conducted since August 1, 2022. Grassley, meanwhile, just turned 89 years old – meaning he could yet stumble due to age.

Lean Republican (n=0).

Tilt Republican (n=0).

Toss-ups (n=3). Here we find three Republican-held Senate seats which Democrats have made the most competitive Senate elections of 2022, although they are currently the barest of underdogs in each. Two seats are those being vacated by Republicans Richard Burr of North Carolina (R+5.8) and Rob Portman of Ohio (R+9.8). On paper, these should be relatively easy holds for Republicans. In North Carolina, though, former State Supreme Court Chief Justice Cheri Beasley is within 0.5 points of House member Ted Budd – and leads by 0.2 points in 10 polls conducted since June 1. Arguably, Beasley’s and Demings’ candidacies have been the most positively impacted by the Dobbs v. Jackson decision overturning the Constitutional right to an abortion. Meanwhile, in Ohio, Republican venture capitalist J.D. Vance trails Democratic House member Tim Ryan 44.5% to 44.3%. The strong caveat, however, is that only six polls rated at least B+ (according to FiveThirtyEight’s pollster ratings) have been conducted thus far, and they give Vance an adjusted average lead of 3.3 points, though even then he still only averages 45.7%.

Luck may finally have run out, meanwhile, for two-term Wisconsin Republican Ron Johnson. While fundamentals cast this election as Tilt Republican, Johnson is essentially tied in seven publicly-available polls (R+0.1) with Democratic Lieutenant Governor Mandela Barnes. However, in the three polls released in the last two weeks (B/B-), Barnes trails Johnson by 2.2 points, after having led by 3.5 points in polls averaging A-.

Tilt Democratic. Warnock leads former National Football League player Herschel Walker 46.7% to 44.9%. However, Georgia is still a Republican-leaning state (R+6.5) and a runoff occasioned by neither candidate winning a majority on Election Day remains a strong possibility. If the race goes to a January runoff, FiveThirtyEight.com projects Walker to win an average 65% of the time. The good news for Warnock, however, is that he has averaged 48.8% in the four publicly-available polls (B/B-) conducted at least partially after Labor Day, with the percentage choosing neither candidate dropping from 9.4 to 6.8.

Being a Democratic incumbent in a purple state (R+0.5) in a neutral environment is why Cortez Masto is projected to win reelection in Nevada by 2.4 points.

Lean Democratic (n=0).

Likely Democratic (n=1). The only reason Democratic Senator Mark Kelly is not a solid/safe choice to defeat venture capitalist Blake Masters is the Republican lean of Arizona (R+6.1). Kelly is already polling at 48.7%, however, and I project him to win by 5.2 points.

Solid Democratic (n=3). The Senate seat most likely to change partisan hands is that held by retiring Republican Pat Toomey in Pennsylvania. Democratic Lieutenant Governor John Fetterman, already averaging 49.3%, holds an 8.5-point lead over television celebrity Dr. Mehmet Oz. While Fetterman’s stroke has complicated this race, it is hard to see how Oz wins. Republican incumbents Michael Bennet of Colorado (D+5.7) and Maggie Hassan of New Hampshire (D+1.2) are clear favorites to win reelection – averaging 47.9% and 49.0%, respectively – likely by high single digits.

Safe Democratic (n=10): Incumbents Richard Blumenthal (CT), Tammy Duckworth (IL), Patty Murray (WA), Padilla, Chuck Schumer (NY), Schatz, Van Hollen and Wyden (OR) are near-locks to win reelection. Murray was effectively reelected on August 2, when she won 52.3% of the vote in an all-candidate primary; all Democrats combined won just under 55% of the vote. Meanwhile, House member Peter Welch is all-but-certain to win the Vermont Senate seat being vacated by seven-term Democrat Pat Leahy.

Bottom line: Democrats will almost certainly add Senate seats, most likely 1-2.

***********

Governors. As of September 19, 2022, 234 publicly-available gubernatorial election polls have been conducted since November 1, 2021. Only those in the open seats in Hawaii and Maryland – which Democrats Josh Green and Wes Moore are near-locks to win – and in Alabama, Idaho, Nebraska, South Dakota, Tennessee, and Wyoming – which Republicans Kay Ivey, Brad Little, Jim Pilllen, Kristi Noem, Bill Lee and Mark Gordon, respectively, are near-locks to win – do not yet have publicly-available polling.

A net gain of three governor’s mansions by Democrats would give them 25, giving them parity for the first time since 2010. Table 2 shows updated values for all 36 gubernatorial elections in 2022.

Table 2: Democratic gubernatorial win probabilities and projected final margins, 2022

StateFundamentalsPolling (WAPA)Poll WtFinal Projection
Democratic seatMarginP(D win)MarginP(D win) MarginP(D win)
Hawaii28.793.7%n/an/an/an/a93.7%
California35.096.9%26.5100.0%0.8327.999.5%
New York19.985.5%13.1100.0%0.8314.397.5%
Rhode Island21.587.3%24.7100.0%0.7423.996.7%
Connecticut24.089.9%10.1100.0%0.9011.599.0%
Illinois23.489.3%13.3100.0%0.7316.097.1%
Oregon9.869.8%2.156.6%0.724.260.2%
New Mexico16.480.7%6.188.4%0.966.588.0%
Colorado15.879.8%8.999.3%0.7910.495.2%
Maine14.678.0%8.395.5%0.749.990.9%
Minnesota11.973.6%9.099.7%0.959.198.3%
Nevada9.669.4%1.359.6%0.812.961.4%
Michigan9.469.0%10.6100.0%0.7510.392.2%
Pennsylvania-2.644.2%8.5100.0%0.856.891.9%
Wisconsin7.765.7%1.661.2%0.902.261.6%
Kansas-11.227.3%6.080.5%0.731.466.2%
Estimated Democratic seats lost = 2.1 (range = 1.5 to 3.1)*
 
Republican seat       
Vermont14.778.2%-49.90.0%0.86-40.811.0%
Maryland25.991.6%n/an/an/an/a91.6%
Massachusetts25.891.5%23.3100.0%0.9123.599.3%
New Hampshire-13.024.2%-19.40.0%0.81-18.24.6%
Florida-19.714.5%-5.60.0%0.79-8.63.1%
Arizona-6.436.4%2.398.9%0.861.064.2%
Georgia-13.723.1%-4.90.0%0.85-6.23.6%
Iowa-24.09.9%-14.50.0%0.84-16.11.6%
Ohio-24.09.9%-12.70.0%0.86-14.31.4%
Texas-26.28.0%-6.90.0%0.81-10.61.5%
Alaska-30.05.4%-11.80.1%0.79-15.51.2%
South Carolina-30.15.3%-7.30.0%0.76-12.71.3%
Nebraska-25.48.6%n/an/an/an/a8.6%
Tennessee-41.41.3%n/an/an/an/a1.3%
Alabama-43.41.0%n/an/an/an/a1.0%
South Dakota-43.80.9%n/an/an/an/a0.9%
Arkansas-30.65.0%-21.10.0%0.42-26.62.9%
Idaho-49.00.4%n/an/an/an/a0.4%
Oklahoma-52.00.3%-12.60.1%0.74-22.70.1%
Wyoming-61.70.0%n/an/an/an/a0.0%
Estimated Republican seats lost = 3.0 (range = 2.6 to 3.4)
Overall expected Democratic seats gained/lost = +0.9 (range = -0.4 to +2.0)

    *Adding or subtracting 3 points to all WAPA, including generic ballot estimates

The gubernatorial incumbency advantages are 10.4 points for Democrats and 13.9 points for Democrats. On average, I now give 18% weight to fundamentals and 82% weight to WAPA, an increase of WAPA weight of 9.0 points in two weeks.

Safe/Solid Republican (n=16). Incumbents Mike DeWine (OH), Mike Dunleavy (AK), Gordon, Ivey, Lee, Little, Henry McMaster (SC), Noem, Kim Reynolds (IA), Kevin Stitt (OK) and Chris Sununu (NH) are near-locks for reelection. University of Nebraska Board of Regents member Pillen and former White House spokesperson Sarah Huckabee Sanders will be the next Republican governors of Nebraska and Arkansas, respectively.

Meanwhile, Democrats Stacey Abrams (GA), Charlie Crist (FL) and Beto O’Rourke (TX) trail incumbent Republican governors Brian Kemp, Ron DeSantis and Greg Abbott, respectively, by an average 5.8 points. The lowest polling average among the three incumbents is Abbott’s 48.2%, though, making it difficult to see how any of these Democrats win.

Likely Republican (n=1). Popular Republican governor Phil Scott will win reelection, period, despite Vermont’s D+28.9 status.

Lean Republican (n=0).

Tilt Republican (n=0).

Toss-ups (n=0).

Tilt Democratic (n=5). The first of three races where Democrats are currently favored, however slightly, to win is Arizona, with Hobbs projected to eke out a 1.0-point victory over Lake. However, it would not take much for this election to move to Toss Up or even Tilt Republican. Balanced on the same precipice is Oregon, though the state’s strong Democratic lean (D+10.1) and the tendency of third-party candidates to fade in polling suggests Kotek will win by mid-to-upper single digits.

This leaves three potentially vulnerable Democratic incumbents who first won in the Democratic 2018 wave: Laura Kelly in Kansas, Steve Sisolak in Nevada and Tony Evers in Wisconsin. Two weeks ago, I thought Kelly was set to lose by 3.8 points, but one post-Labor-Day Echelon Insights poll (B/C) giving her a 12-point lead bumped her average to 49.5%, putting her on the doorstep of winning in this R+21.3 state. The good news for Evers is that he is already averaging 48.0%, though he is only 1.6 points ahead of businessman Tim Michels. Sisolak, however, is only averaging 43.3% to Clark County Sheriff Joe Lombardo’s 42.0%. Still, the incumbency advantage for Democrats is likely strong enough to win them both a second term.

Lean Democratic (n=0).

Likely Democratic (n=1). The only reason New Mexico Democratic Governor Michelle Lujan Grisham is not a solid/safe choice to defeat Republican former meteorologist Mark Ronchetti is that she is averaging 47.1% in seven very high-quality (A- average) publicly-available polls. I project Lujan Grisham to win by mid-to-high single digits.

Solid/Safe Democratic (n=13). The other states which Democrats are near-locks to flip are Maryland and Massachusetts, with both former Army Captain Moore and state Attorney General Maura Healey projected to win by more than 20 points. Two other Democratic statewide officials are heavy favorites to win governor’s mansions held by retiring Democrats: Lieutenant Governor Green in Hawaii and Attorney General Josh Shapiro in Pennsylvania.

That leaves eight incumbent Democrats who are at least 9-1 favorites to win reelection: Kathy Hochul (NY), Ned Lamont (CT), Dan McKee (RI), Janet Mills (ME), Gavin Newsom (CA), Jared Polis (CO), J.B. Pritzker (IL), Tim Walz (MN) and Gretchen Whitmer (MI)

Bottom line: There will almost certainly be more Democratic governors, most likely 1-2

***********

While Democrats find themselves in a very good position right now – poised to gain anywhere from one to four Senate seats and from one to three governor’s mansions while limiting losses in the House to ~10 seats – it would only take a not-historically-unusual polling miss of 3-4 points in Republicans’ favor to wipe away these gains. In this scenario, Democrats still flip Pennsylvania, while losing in Georgia and Nevada for a net loss of one seat and control of the Senate. Democrats would still win back governor’s mansions in Maryland and Massachusetts, while losing in Kansas, Nevada and Wisconsin, for a net loss of one governor’s mansion. And the likelihood of retaining House control would drop to 2.8%, centered around a loss of 24-25 seats.

This is what would happen if undecideds broke 2-1 Republican across the board, in line with midterm election history. However, it is equally plausible Democrats overperform polls by a similar amount. In this scenario, Democrats would be slight favorites (62.2%) to retain House control, with an identical 222-213 split as after the 2020 elections. Not only would Democrats flip Pennsylvania and Wisconsin while holding all their Senate seats, they would also add North Carolina and Ohio – and, perhaps, Florida; this would be a gain of 4-5 seats. They would still top out at netting three governor’s mansions, though Georgia might creep into play.

As of mid-September, though, I prefer to drive straight down the middle, with Democrats limiting House losses to ~10 seats while netting one-two Senate seats and one-three governor’s mansions, outcomes which seemed like pipe dreams even six months ago.

Early voting starts in a few weeks.

Until next time, please wear a mask as necessary to protect yourself and others – and if you have not already done so, get vaccinated against COVID-19! Also, if you are not already registered to vote, please do so immediately. And if you like what you read on this website, please consider making a donation. Thank you.

One thought on “2022 Elections Update: Rethinking polling margins of error

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s