The primary inspiration for this website was FiveThirtyEight.com. When I started to collect and analyze election polling data in earnest early in 2018, I followed its lead by developing my own polling aggregation: WAPA (weighted-adjusted polling average). I adjusted/weighted polls for recency, pollster quality, partisan lean, whether only adults were queried and whether percentages summed to 100% (i.e., no undecided or third-party voters).
Pollster quality – essentially, adherence to gold-standard methodology – was routinely calculated by FiveThirtyEight.com, first as a letter grade (A, B, C, etc.) then as a value between 0.0 and 3.0. With the closure of FiveThirtyEight.com, however, these pollster ratings no longer exist. Nor is there a single online location for every newly-released election poll, though Ballotpedia may prove sufficient.
As a result, I once again rethought how I aggregate polls. And, as always, unless otherwise specified, election data come from Dave Leip’s indispensable website.
***********
I started by assessing how accurately WAPA “predicted” the 2024 elections, starting with the generic ballot.[1] The final WAPA showed Democrats leading Republicans nationally 46.4% to 45.7%, or by 0.7 percentage points (“points”). However, Democrats actually lost all votes cast nationwide for the United States House of Representatives (“House”) by 2.7 points: 47.8% to 50.6%. WAPA thus overestimated Democratic strength in House elections by 3.4 points. More to the point, Republicans overperformed their final WAPA percentage by 4.9 points, compared to 1.4 points for Democrats, with the typical 1.5 points for third-party candidates.
In the presidential election, meanwhile, the final WAPA showed Democratic Vice President Kamala Harris leading former Republican president Donald Trump nationally 48.4% to 46.6%, or by 1.8 points. Instead, Harris lost nationally 48.2% to 49.7%, or by 1.5 points. WAPA thus overestimated Democratic strength in presidential voting nationally by 3.3 points. While the final WAPA came within 0.2 points of Harris’ national percentage, it underestimated Trump’s percentage by 3.1 points.
Table 1 compares final WAPA with the final Harris-Trump margins in all 50 states and the District of Columbia. “ABS” stands for “absolute value,” the difference between WAPA and final Harris-Trump margin regardless of direction. Swing states are in bold.
Table 1: WAPA and final margins by state in the 2024 presidential election:
| State | Harris-Trump margin | Difference | ABS Diff | |
| WAPA | Final | |||
| Oregon | 10.4 | 14.3 | 3.9 | 3.9 |
| Washington | 17.6 | 18.2 | 0.6 | 0.6 |
| Utah | -21.7 | -21.2 | 0.5 | 0.5 |
| Maryland | 29.1 | 28.5 | -0.6 | 0.6 |
| Georgia | -1.5 | -2.2 | -0.7 | 0.7 |
| Virginia | 6.6 | 5.8 | -0.8 | 0.8 |
| New Mexico | 7.5 | 6.0 | -1.5 | 1.5 |
| Colorado | 12.5 | 11.0 | -1.5 | 1.5 |
| Connecticut | 16.2 | 14.5 | -1.7 | 1.7 |
| Massachusetts | 26.9 | 25.2 | -1.7 | 1.7 |
| Montana | -18.1 | -19.9 | -1.8 | 1.8 |
| Minnesota | 6.2 | 4.2 | -2.0 | 2.0 |
| Oklahoma | -32.1 | -34.3 | -2.2 | 2.2 |
| North Carolina | -1.0 | -3.2 | -2.2 | 2.2 |
| Rhode Island | 16.0 | 13.8 | -2.2 | 2.2 |
| Maine | 9.2 | 6.9 | -2.3 | 2.3 |
| Wisconsin | 1.4 | -0.9 | -2.3 | 2.3 |
| Pennsylvania | 0.7 | -1.7 | -2.4 | 2.4 |
| DC | 86.3 | 83.8 | -2.5 | 2.5 |
| Delaware | 17.7 | 14.7 | -3.0 | 3.0 |
| Michigan | 1.6 | -1.4 | -3.0 | 3.0 |
| Indiana | -15.8 | -18.9 | -3.1 | 3.1 |
| Nevada | 0.2 | -3.1 | -3.3 | 3.3 |
| Louisiana | -18.7 | -22.0 | -3.3 | 3.3 |
| Arizona | -2.0 | -5.5 | -3.5 | 3.5 |
| Ohio | -7.6 | -11.1 | -3.5 | 3.5 |
| South Dakota | -25.5 | -29.2 | -3.7 | 3.7 |
| Kentucky | -26.7 | -30.5 | -3.8 | 3.8 |
| Missouri | -13.8 | -18.4 | -4.6 | 4.6 |
| Nebraska | -15.8 | -20.5 | -4.7 | 4.7 |
| New Hampshire | 7.5 | 2.8 | -4.7 | 4.7 |
| Alaska | -8.4 | -13.1 | -4.7 | 4.7 |
| Alabama | -25.6 | -30.5 | -4.9 | 4.9 |
| New York | 17.5 | 12.5 | -5.0 | 5.0 |
| Wyoming | -40.7 | -45.8 | -5.1 | 5.1 |
| Idaho | -31.2 | -36.4 | -5.2 | 5.2 |
| South Carolina | -12.3 | -17.9 | -5.6 | 5.6 |
| Vermont | 37.5 | 31.5 | -6.0 | 6.0 |
| Tennessee | -23.4 | -29.6 | -6.2 | 6.2 |
| Illinois | 17.1 | 10.9 | -6.2 | 6.2 |
| Kansas | -9.6 | -16.0 | -6.4 | 6.4 |
| Texas | -7.2 | -13.7 | -6.5 | 6.5 |
| Mississippi | -16.1 | -22.9 | -6.8 | 6.8 |
| California | 27.1 | 20.1 | -7.0 | 7.0 |
| Iowa | -6.0 | -13.2 | -7.2 | 7.2 |
| Florida | -5.5 | -13.1 | -7.6 | 7.6 |
| Hawaii | 32.6 | 23.1 | -9.5 | 9.5 |
| West Virginia | -30.4 | -41.0 | -10.6 | 10.6 |
| New Jersey | 16.5 | 5.9 | -10.6 | 10.6 |
| Arkansas | -19.6 | -30.6 | -11.0 | 11.0 |
| North Dakota | -23.9 | -36.5 | -12.6 | 12.6 |
| AVERAGE | -0.8 | -4.9 | -4.2 | 4.4 |
The final WAPA was more inaccurate at the state level, overestimating Harris’ margin by an average 4.2 points – and 4.4 points when direction is ignored. In fact, the final WAPA underestimated Harris’ margin in only three western states, Oregon, Utah and Washington – and only in Oregon by more than 1.0 points. Curiously, WAPA fared better in the seven swing states (all won by Trump), overestimating Harris’ margin by “only” 2.5 points. The final WAPA, albeit calculated slightly differently, was more accurate at the state level in 2020, with an average Democratic overestimation of 3.2 points – 4.3 points if direction is ignored.
Table 2, meanwhile, compares the final WAPA to final margins (Democratic % minus Republican %) in the 34 elections for United States Senate (“Senate”) and 11 elections for governor in 2024.[2]
Table 2: WAPA and final margins in 2024 Senate and gubernatorial elections:
| State | Election | D-R margin | Difference | ABS Diff | |
| WAPA | Final | ||||
| Rhode Island | Senate | 15.3 | 20.0 | 4.7 | 4.7 |
| North Carolina | Governor | 11.8 | 14.8 | 3.0 | 3.0 |
| Connecticut | Senate | 16.1 | 18.9 | 2.8 | 2.8 |
| Montana | Governor | -21.8 | -20.2 | 1.6 | 1.6 |
| Minnesota | Senate | 14.3 | 15.7 | 1.4 | 1.4 |
| Hawaii | Senate | 32.4 | 32.7 | 0.3 | 0.3 |
| Montana | Senate | -7.1 | -7.1 | 0.0 | 0.0 |
| Utah | Governor | -24.1 | -24.4 | -0.3 | 0.3 |
| Nebraska | Senate | -6.2 | -6.7 | -0.5 | 0.5 |
| New Mexico | Senate | 10.7 | 10.1 | -0.6 | 0.6 |
| Washington | Senate | 19.8 | 18.5 | -1.3 | 1.3 |
| Washington | Governor | 13.3 | 11.2 | -2.1 | 2.1 |
| Virginia | Senate | 11.1 | 8.9 | -2.2 | 2.2 |
| Wisconsin | Senate | 3.1 | 0.8 | -2.3 | 2.3 |
| Arizona | Senate | 5.0 | 2.4 | -2.6 | 2.6 |
| Indiana | Governor | -10.6 | -13.3 | -2.7 | 2.7 |
| Massachusetts | Senate | 22.5 | 19.8 | -2.7 | 2.7 |
| Missouri | Senate | -11.0 | -13.7 | -2.7 | 2.7 |
| Nevada | Senate | 5.0 | 1.6 | -3.4 | 3.4 |
| Wyoming | Senate | -47.3 | -51.0 | -3.7 | 3.7 |
| Maine | Senate | 21.2 | 17.4 | -3.8 | 3.8 |
| Ohio | Senate | 0.2 | -3.6 | -3.8 | 3.8 |
| Delaware | Senate | 21.0 | 17.1 | -3.9 | 3.9 |
| Pennsylvania | Senate | 3.7 | -0.2 | -3.9 | 3.9 |
| Michigan | Senate | 4.5 | 0.3 | -4.2 | 4.2 |
| Texas | Senate | -4.3 | -8.5 | -4.2 | 4.2 |
| Maryland | Senate | 16.4 | 11.8 | -4.6 | 4.6 |
| Utah | Senate | -25.8 | -30.6 | -4.8 | 4.8 |
| California | Senate | 23.6 | 17.7 | -5.9 | 5.9 |
| New York | Senate | 24.2 | 18.3 | -5.9 | 5.9 |
| Indiana | Senate | -13.5 | -19.9 | -6.4 | 6.4 |
| Vermont | Senate | 37.6 | 31.1 | -6.5 | 6.5 |
| Missouri | Governor | -13.7 | -20.4 | -6.7 | 6.7 |
| North Dakota | Senate | -26.0 | -32.9 | -6.9 | 6.9 |
| Nebraska Special | Senate | -17.9 | -25.1 | -7.2 | 7.2 |
| West Virginia | Senate | -33.5 | -41.0 | -7.5 | 7.5 |
| Florida | Senate | -4.9 | -12.8 | -7.9 | 7.9 |
| Tennessee | Senate | -21.1 | -29.6 | -8.5 | 8.5 |
| New Jersey | Senate | 18.9 | 9.6 | -9.3 | 9.3 |
| New Hampshire | Governor | 0.0 | -9.3 | -9.3 | 9.3 |
| Delaware | Governor | 21.9 | 12.1 | -9.8 | 9.8 |
| West Virginia | Governor | -20.0 | -30.4 | -10.4 | 10.4 |
| North Dakota | Governor | -31.3 | -42.3 | -11.0 | 11.0 |
| Vermont | Governor | -37.3 | -51.6 | -14.3 | 14.3 |
| Mississippi | Senate | -10.3 | -25.6 | -15.3 | 15.3 |
| AVERAGE | -0.3 | -4.6 | -4.3 | 4.9 | |
| AVERAGE Senate | 2.9 | -1.0 | -3.9 | 4.5 | |
| AVERAGE Governor | -6.9 | -12.0 | -5.2 | 5.9 | |
Across these 45 elections, the final WAPA overestimated Democratic margins by an average 4.3 points – and 4.9 points regardless of direction. WAPA fared slightly better in Senate elections than in gubernatorial election, with an average Democratic overestimation of 3.9 points (4.5 points disregarding direction) in the former and 5.2 points (5.9 points) in the latter. One possible explanation is that Senate races were polled more twice as often as governor’s races. The median number of polls in the 34 Senate elections was nine, ranging from 0 in Delaware to 127 in Pennsylvania, while the median number in gubernatorial elections was four, ranging from two in four states (Delaware, North Dakota, Vermont, West Virginia) to 86 in North Carolina.[3]
Average difference between polling averages and election margins of between 3.3 and 5.2 points is well within historic parameters. What is unusual, though, is that WAPA underestimated Democratic margins in only 10 of 96 (10%) state-level elections in 2024. Three states – medium blue New Jersey and deep red North Dakota and West Virginia – saw multiple Democratic margin overestimations of >9.0 points.
Some random deviation, even after averaging across many polls, from the final margin is to be expected. But that deviation typically features a high number of misses in both directions. This is what happened in 2022, when the average difference between final WAPA and final margin across 69 Senate and gubernatorial elections was a Democratic overestimation of just 1.8 points – with a much higher 42% (27 of 64) featuring a Democratic underestimation.[4]
In other words, differences between final WAPA and final margins in 2024 were systematic, not random. It resembles what happened in 2020 enough that I hypothesize polling underestimates Republican strength across the board by about 2.5 points when Trump himself is on the ballot. There is insufficient data, however, to do more than speculate.
***********
Having assessed WAPA’s relative accuracy, is there any way I can improve its predictive value, outside of improving polling itself (and, hypothetically, removing Trump from the ballot)?
Maybe.
A total of 233 polls assessed the generic ballot between January 1 and Election Day in 2024. This is nearly half the 442 generic ballot polls conducted during the same period in 2022. Fully 205 (88%) of the 2024 generic ballot polls were conducted by pollsters who assessed the generic ballot at least three times. Their final time-weighted averages are listed in Table 3.
Table 3: Time-weighted generic ballot polling averages in 2024 by pollster
| Pollster | # Polls | Dem% | Rep% | D%-R% | Diff | 538 Rating[5] |
| McLaughlin & Associates | 13 | 44.0 | 47.4 | -3.4 | -0.7 | 0.5 |
| Fabrizio, Lee & Associates/ GBAO | 5 | 45.1 | 48.2 | -3.1 | -0.4 | 1.5 |
| OnMessage | 17 | 43.0 | 45.9 | -2.9 | -0.2 | 1.1 |
| NewsNation | 4 | 43.2 | 44.3 | -1.1 | 1.6 | 1.6 |
| Cygnal | 10 | 47.0 | 47.8 | -0.8 | 1.9 | 2.1 |
| HarrisX | 14 | 45.6 | 46.1 | -0.5 | 2.2 | 1.6 |
| Noble Predictive Insights | 5 | 45.7 | 46.1 | -0.4 | 2.3 | 1.8 |
| Hart Research Associates/ Public Opinion Strategies | 8 | 46.3 | 46.7 | -0.4 | 2.3 | 2.6 |
| RMG Research | 6 | 46.9 | 47.1 | -0.2 | 2.5 | 2.3 |
| Big Data Poll | 3 | 44.9 | 44.6 | 0.3 | 3.0 | 1.4 |
| Echelon Insights | 11 | 47.8 | 47.1 | 0.7 | 3.4 | 2.7 |
| Florida Atlantic University Pol Com Lab/ Mainstreet Research | 6 | 46.0 | 44.9 | 1.1 | 3.8 | 1.4 |
| Monmouth University Polling Institute | 4 | 47.2 | 46.0 | 1.2 | 3.9 | 2.9 |
| Marist College | 6 | 48.1 | 46.8 | 1.3 | 4.0 | 2.9 |
| Emerson College | 15 | 47.4 | 46.0 | 1.4 | 4.1 | 2.9 |
| YouGov | 55 | 46.3 | 44.7 | 1.6 | 4.3 | 2.9 |
| The Bullfinch Group | 3 | 43.8 | 42.2 | 1.6 | 4.3 | 1.4 |
| Marquette University School of Law | 4 | 50.9 | 49.1 | 1.8 | 4.5 | 3.0 |
| Suffolk University | 3 | 46.8 | 45.0 | 1.8 | 4.5 | 2.9 |
| Clarity Campaign Labs | 9 | 45.1 | 42.2 | 2.9 | 5.6 | 1.6 |
| Data For Progress | 4 | 47.7 | 44.7 | 3.0 | 5.7 | 2.7 |
| Average | 205 | 46.1 | 45.9 | 0.3 | 3.0 | 2.1 |
As a group, these 21 pollsters were a bit more accurate, overestimating the national Democratic House margin by 3.0 points. Still, only three (14%) underestimated Democratic strength, all by less than 0.8 points. Curiously, these pollsters – McLaughlin & Associates; Fabrizio, Lee & Associates/GBAO; OnMessage – had a strong Republican lean with a low FiveThirtyEight.com rating, averaging the equivalent of a C-/D+. By contrast, the six nonpartisan pollsters with a rating of at least 2.9 overestimated the Democratic margin by an average 4.2 points. In fact, the higher the pollster quality, the higher the deviation in measuring the 2024 generic ballot (r=0.63). Thus, adjusting for pollster quality likely made the final 2024 generic ballot WAPA less accurate.
In 2022, meanwhile, the 38 pollsters who assessed the generic ballot at least three times overestimated the final national Democratic House margin by just 2.2 points, with seven (18%) underestimating this margin. That year, however, there was no association between pollster quality and accuracy (r=0.07).
The story is the same for assessing the national Harris-Trump vote in 2024, as Table 4 shows. Because I calculated separate WAPA for polls with and without third-party candidates, Table 4 lists the average (weighted by number of polls) of the two values.
Table 4: Time-weighted Harris-Trump national polling averages in 2024 by pollster
| Pollster | # Polls | Harris | Trump | H-T | Diff | 538 Rating |
| McLaughlin & Associates | 6 | 44.4 | 47.4 | -3.0 | -1.6 | 0.5 |
| AtlasIntel | 7 | 47.0 | 49.9 | -2.9 | -1.5 | 2.7 |
| OnMessage | 3 | 45.5 | 46.9 | -1.5 | 0.0 | 1.1 |
| SoCal Research | 5 | 48.0 | 49.3 | -1.3 | 0.1 | 1.4 |
| SurveyUSA | 3 | 44.4 | 45.1 | -0.7 | 0.8 | 2.8 |
| Beacon Research/ Shaw & Company Research | 5 | 48.5 | 49.2 | -0.7 | 0.8 | 2.8 |
| Quinnipiac University | 3 | 47.5 | 48.0 | -0.5 | 1.0 | 2.8 |
| Big Data Poll | 3 | 48.5 | 48.9 | -0.5 | 1.0 | 1.4 |
| Siena College/New York Times | 6 | 46.1 | 46.3 | -0.2 | 1.2 | 3.0 |
| Quantus Polls and News | 4 | 48.5 | 48.4 | 0.0 | 1.5 | 1.4 |
| SSRS | 3 | 47.1 | 47.0 | 0.1 | 1.5 | 2.1 |
| Hart Research Associates/ Public Opinion Strategies | 6 | 46.5 | 46.2 | 0.3 | 1.7 | 2.6 |
| HarrisX | 18 | 48.2 | 47.9 | 0.3 | 1.7 | 1.6 |
| ActiVote | 14 | 50.2 | 49.8 | 0.4 | 1.9 | 1.4 |
| Emerson College | 8 | 49.3 | 48.8 | 0.5 | 2.0 | 2.9 |
| Redfield & Wilton Strategies | 11 | 46.1 | 45.1 | 1.0 | 2.5 | 1.8 |
| Change Research | 4 | 46.8 | 45.8 | 1.1 | 2.5 | 1.4 |
| YouGov | 39 | 47.4 | 45.7 | 1.7 | 3.1 | 2.9 |
| TIPP Insights | 14 | 47.8 | 46.1 | 1.7 | 3.2 | 1.8 |
| RMG Research | 14 | 49.0 | 47.2 | 1.7 | 3.2 | 2.3 |
| Leger | 4 | 49.7 | 47.6 | 2.0 | 3.5 | 2.0 |
| Cygnal | 7 | 47.7 | 45.6 | 2.1 | 3.6 | 2.1 |
| Florida Atlantic University PolCom Lab/ Mainstreet Research | 5 | 48.6 | 46.5 | 2.1 | 3.6 | 1.4 |
| Noble Predictive Insights | 3 | 48.6 | 46.2 | 2.5 | 3.9 | 2.4 |
| Angus Reid Global | 4 | 48.4 | 45.7 | 2.7 | 4.2 | 2.0 |
| Ipsos | 24 | 47.8 | 44.8 | 3.0 | 4.5 | 2.8 |
| Marist College | 7 | 50.6 | 47.4 | 3.2 | 4.7 | 2.9 |
| Echelon Insights | 7 | 49.7 | 46.4 | 3.3 | 4.7 | 2.7 |
| Data For Progress | 4 | 49.1 | 45.8 | 3.3 | 4.8 | 2.7 |
| Morning Consult | 38 | 49.6 | 45.7 | 3.9 | 5.4 | 1.8 |
| Big Village | 11 | 47.8 | 43.7 | 4.1 | 5.6 | 1.6 |
| Clarity Campaign Labs | 4 | 50.1 | 45.6 | 4.6 | 6.1 | 1.6 |
| Outward Intelligence | 11 | 51.5 | 45.7 | 5.9 | 7.4 | 1.4 |
| Average | 305 | 48.1 | 46.8 | 1.2 | 2.7 | 2.1 |
These 33 pollsters accounted for 81% of the 375 polls conducted nationally of the Harris-Trump election in 2024. They were again slightly more accurate, overestimating the final national Harris-Trump margin by just 2.7 points. Only two of these 33 pollsters (6%) underestimated this margin, with low-rated Republican pollsters OnMessage and SoCalStrategies hitting it almost exactly. The correlation between pollster quality and deviation from the margin was only 0.08 – suggesting that being a Republican-leaning pollster was a better predictor of accuracy than adhering to gold-standard methodology.
***********
I did not formally assess other adjustments, other than to observe Republican-leaning OnMessage was among the most accurate pollster in 2024, as was AtlasIntel, who queries all adults, while ActiVote and Marquette University Law School, whose percentages total 100%, had an average final Democratic margin overestimation of 3.3 points – right on the average. One conclusion, then, is that the only adjustment besides time worth doing is giving more weight to Republican pollsters. Except, nonpartisan pollsters SurveyUSA, Beacon Research/Shaw & Company Research (Fox News), Quinnipiac University and Siena College/New York Times only overestimated the final Harris-Trump margin by 1.0 points, on average.
Thus, when I recalculated 2024 WAPA for generic ballot and national Harris-Trump margin, I only adjusted for recency. For the former, Democrats led 45.84% to 45.65%, or 0.2 points, which is a Democratic overestimation of 2.9 points – an improvement of 0.5 points. And for the latter, percentages barely changed: 48.2% for Harris, 46.6% for Trump, an improvement of 0.2 points.
Going forward, then, I will only adjust polls for recency. I will also change the name of my polling aggregation metric to RWPA, or recency-weighted polling average. I will not know for a year whether RWPA is more (or less) accurate than WAPA, but it will certainly be far less time-consuming to calculate.
Until next time…and if you like what you read here, please consider making a donation. Thank you!
[1] A variation of “If the election in your district were held today, would you vote for the Democrat, the Republican, someone else, or nobody?”
[2] I counted Independents Angus King of Maine, Dan Osborn of Nebraska and Bernie Sanders of Vermont as “Democrats.”
[3] Averages, heavily skewed by outliers, are 23.5 (Senate) and 11.0 (governor).
[4] Five elections were not polled: Hawaii Senate and governor, Idaho Senate and governor, North Dakota Senate.
[5] I assigned a 1.4 – the equivalent of a C+ – to pollsters with no FiveThirtyEight.com rating.
Just Bear With Me... 
One thought on “Assessing polling accuracy in 2024”