2020 South Carolina Primary: Final Polling Update

From 7 am to 7 pm EST on February 29, 2020, polls will be open in South Carolina for that state’s Democratic presidential primary. This election follows similar contests in Iowa and New Hampshire in which United States Senator (“Senator”) from Vermont Bernie Sanders and former South Bend, Indiana Mayor Pete Buttigieg fared very well, and a strong win by Sanders in the Nevada Caucuses–where former Vice President Joe Biden finished a strong second. Unlike those three states, though, the Democratic primary electorate in South Carolina is expected to be majority-black; three in five (61%) 2016 South Carolina Democratic Primary voters were black, as were 55% in 2008. This is also the final early nomination-related election in which former New York City Mayor Michael Bloomberg will not appear on the ballot.

To learn how I calculate candidate WAPA (weighted-adjusted polling average), please see here. Here is my updated weighting scheme:

  • Polls conducted entirely or partially after February 3, 2020, but before February 12, 2020 are weighted 2.00 or 1.00+fraction[1] times, respectively, higher than polls conducted entirely before February 4, 2020.
  • Polls conducted entirely or partially after February 11, 2020, but before February 23, 2020 are weighted 3.00 or 2.00+fraction times, respectively, higher than polls conducted entirely before February 12, 2020.
  • Polls conducted entirely or partially after February 22, 2020, but before March 1, 2020 are weighted 4.00 or 3.00+fraction times, respectively, higher than polls conducted entirely before February 23, 2020.

I have only been to South Carolina once, driving through it one March 1990 day as part of an epic round-trip road trip between Boston, Massachusetts and Atlanta, Georgia. I have no photographs from that trip, however, so here is one I took in Concord, Massachusetts in October 2008 of trees mimicking a traffic light.

Traffic light trees 2 10-12-2008

**********

Here is a breakdown of publicly-available South Carolina Primary polls as of midnight EST on February 29, 2020:

  • 53 since January 1, 2019
  • 41 since the 1st Democratic debate on June 26, 2019
  • 22 since the 5th Democratic debate on November 19, 2019
  • 18 since the 7th Democratic debate on January 14, 2020
  • 15 since the Iowa Caucuses on February 3, 2020
    • 7 between February 12 and February 22
    • 8 beginning February 23

Table 1: Final South Carolina WAPA for declared 2020 Democratic presidential nomination candidates

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Since

Iowa

Biden 33.2 33.0 32.4 32.2 32.3
Sanders 18.6 18.7 20.3 20.7 20.7
Steyer 11.4 11.7 13.4 13.5 13.5
Warren 9.1 9.1 8.4 8.1 8.1
Buttigieg 7.7 7.7 8.5 8.7 8.8
Klobuchar 3.9 4.0 4.6 4.8 4.9
Gabbard 2.1 2.2 2.5 2.6 2.4
Bloomberg 0.3 0.3 0.3 0.2 0.2
DK/Other 13.7 13.3 9.6 9.1 9.1

Other than a slight uptick for Sanders and billionaire activist Tom Steyer—and an unsurprising reduction in the percentage either undecided or choosing an unlisted candidate—it is difficult to discern momentum in either direction for any candidate in Table 1. Biden has consistently led polls of the South Carolina Primary, hovering in the 32-34% range on average, with Sanders in second place at 18-21%. If these numbers are predictive, only Biden, Sanders and (possibly) Steyer seem poised to crack the 15% statewide threshold to be awarded pledged delegates to the Democratic National Convention this July; depending on how their support is distributed geographically, Massachusetts Senator Elizabeth Warren and Buttigieg could accrue some delegates in one or more Congressional districts.

However, the percentages calculated since the Iowa Democratic Caucuses on February 3 mask a substantial shift in support following the Nevada Democratic Caucuses on February 22 (Table 2).

Table 2: South Carolina WAPA for declared 2020 Democratic presidential nomination candidates following the Iowa Democratic Caucuses

Candidate Before Nevada Caucuses After Nevada Caucuses Change
Biden 26.0 36.5 +10.5
Sanders 21.2 20.6 -0.6
Steyer 15.7 12.9 -2.8
Warren 8.7 7.7 -1.0
Buttigieg 9.5 8.3 -1.2
Klobuchar 5.7 4.0 -1.8
Gabbard 2.1 2.2 +0.1
Bloomberg 0.7 0.0 -0.7
DK/Other 10.5 7.8 -2.7

After finishing 4th and 5th in the Iowa Caucuses and New Hampshire Primary, respectively, Biden appeared to be teetering on the edge of irrelevance, falling behind Sanders in my overall WAPA for the first time (data not shown). But his strong second-place finish in the Nevada Caucuses may have been just what Biden needed: his standing has soared more than 10 percentage points (“points”) in the South Carolina Primary since then. The percentage undecided or naming other candidates has dropped nearly three points over the same time span; Biden could well be picking up support from these late-deciders. At the same time, drops in support of between 1.2 and 2.8 points were registered by Steyer, Minnesota Senator Amy Klobuchar and Buttigieg .

One slight note of caution: using FiveThirtyEight.com’s pollster ratings, the eight pollsters assessing South Carolina Democratic voters before the Nevada Caucuses had a B average, while the seven pollsters in the field afterward had a B- average. Still, if this late shift in support in Biden is real—and it continues as voters make up their minds—Biden could earn 40% of the vote on Saturday, with Sanders the only other candidate to top 20%. The question then would be whether any other candidates accrue any delegates.

We shall see.

Until next time…

[1] Percentage of days the poll was being conducted were after the most recent primary or caucuses

2020 Nevada Caucuses: How did my final polling averages fare?

Given the extremely volatile polling for the 2020 Democratic presidential nomination now that voting has commenced, I will not provide global monthly updates for the next few months. Instead, I will focus on the first handful of primaries and caucuses: Iowa on February 3, New Hampshire on February 11, Nevada on February 22, South Carolina on February 20, the 14 Super Tuesday contests on March 3, and so forth.

Here is my updated weighting scheme:

  • Polls conducted entirely or partially after February 3, 2020, but before February 12, 2020 are weighted 2.00 or 1.00+fraction[1] times, respectively, higher than polls conducted entirely before February 4, 2020.
  • Polls conducted entirely or partially after February 11, 2020, but before February 23, 2020 are weighted 3.00 or 2.00+fraction times, respectively, higher than polls conducted entirely before February 12, 2020.
  • Polls conducted entirely or partially after February 22, 2020, but before March 1, 2020 are weighted 4.00 or 3.00+fraction times, respectively, higher than polls conducted entirely before February 23, 2020.

Table 1 below lists my final Nevada Caucuses WAPA (weighted-adjusted polling average) for the eight declared Democratic presidential candidates, calculated four different ways:

  • Using all 22 polls conducted since January 1, 2019
  • Using only the 19 polls released since the 1st Democratic debate on June 26, 2019
  • Using only the 10 polls released since the 5th Democratic debate on November 19, 2019
  • Using only the 7 polls released since the 7th Democratic debate on January 14, 2020

Feb 2020 lighthouse

Table 1: Final Nevada Caucuses WAPA for declared 2020 Democratic presidential nomination candidates

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate
Sanders 24.6 24.8 26.4 27.8
Biden 19.6 19.0 16.8 15.5
Warren 13.4 13.4 11.4 11.2
Buttigieg 10.9 11.0 12.6 13.7
Steyer 9.8 9.8 12.2 12.9
Klobuchar 6.5 6.3 8.3 9.4
Gabbard 1.4 1.3 1.5 1.5
Bloomberg 0.1 0.1 0.1 0.0
DK/Other 13.7 14.4 10.6 7.9

Based solely on these numbers, one could reasonably draw the following conclusions:

  • United States Senator (“Senator”) from Vermont Bernie Sanders; former South Bend, Indiana Mayor Pete Buttigieg; Minnesota Senator Amy Klobuchar and billionaire activist Tom Steyer were rising in the polls heading into the Nevada Caucuses.
  • Former Vice President Joe Biden and Massachusetts Senator Elizabeth Warren were declining in the polls.
  • No other candidate was moving in the polls in either direction.

Unlike results from the Iowa Democratic Caucuses conducted 19 days earlier, the following three unofficial, final caucus tabulations were reported by the Nevada Democratic Party within two days (Table 2), though not without some griping:

  1. Initial headcount of support for each Democratic candidate (“Initial tally”)
  2. Post-realignment headcount of support for each Democratic candidate (“Final tally”)
  3. Allocation of “county convention delegates,” or CCD’s, the only measure previously reported [ Note: I mistakenly referred to these as “state delegate equivalents” in the earlier post.]

Table 2: Final Nevada Democratic Caucuses results, February 3, 2020

Candidate Initial Tally Final Tally SDE’s
Sanders 34.0 40.5 46.8
Biden 17.6 18.9 20.2
Warren 12.8 11.5 9.7
Buttigieg 15.4 17.3 14.3
Steyer 9.1 4.1 4.7
Klobuchar 9.6 7.3 4.2
Gabbard 0.3 0.0 0.1
Bloomberg 0.0 0.0 0.0

The following three tables list the arithmetic differences between each candidate’s final Nevada Caucuses WAPA and each of the three reported measures; positive values indicate better performance in the Caucuses than in the polls.

Table 3: Arithmetic difference between Initial Nevada Caucuses % of vote and Nevada Caucuses WAPA

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Mean

Difference

Sanders 9.4 9.2 7.6 6.2 8.1
Biden -2.0 -1.4 0.8 2.1 -0.2
Warren -0.6 -0.6 1.4 1.6 0.5
Buttigieg 4.5 4.4 2.8 1.7 3.3
Steyer -0.7 -0.7 -3.1 -3.8 -2.1
Klobuchar 3.1 3.3 1.3 0.2 2.0
Gabbard -1.1 -1.0 -1.2 -1.2 -1.1
Bloomberg -0.1 -0.1 -0.1 0.0 -0.1

Initial tally. If the Nevada Caucuses were instead the Nevada Primary, this would have been the only vote reported. On this measure Sanders, Buttigieg and Klobuchar outperformed their final WAPA percentages by 8.1, 3.3 and 2.0 percentage points (“points”), respectively. And for these three candidates, the closer in time the polls were to the Caucuses, the more “accurate” the WAPA. By contrast, Steyer performed an average of 2.1 points worse in the initial tally than his WAPA, with his percentages becoming less accurate over time. United House of Representatives Member from Hawaii Tulsi Gabbard also performed worse than her WAPA, with no noticeable effect of polling period. Finally, WAPA for Biden and Warren were very close, on average, to their initial tallies, with a minimal impact of polling period.

The overperformance of Sanders and Buttigieg aside, these polls were remarkably accurate. This is reassuring given the dearth and quality of recent Nevada Caucuses polling I detailed in the previous post.

Table 4: Arithmetic difference between Final Nevada Caucuses % of vote and Nevada Caucuses WAPA

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Mean

Difference

Sanders 15.9 15.7 14.1 12.7 14.6
Biden -0.7 -0.1 2.1 3.4 1.2
Warren -1.9 -1.9 0.1 0.3 -0.8
Buttigieg 6.4 6.3 4.7 3.6 5.3
Steyer -5.7 -5.7 -8.1 -8.8 -7.1
Klobuchar 0.8 1.0 -1.0 -2.1 -0.4
Gabbard -1.4 -1.3 -1.5 -1.5 -1.4
Bloomberg -0.1 -0.1 -0.1 0.0 -0.1

Final tally. Only three candidates improved their vote totals after supporters of non-viable candidates shifted to a viable candidate (15% of attendees at a precinct caucus):

  • Sanders (+5,423 supporters; +6.5 points)
  • Buttigieg (+1,496; +1.9)
  • Biden (755; +1.3)

These three candidates also performed better in the final tally than their WAPA, on average; Sanders, in particular, outperformed these WAPA by a remarkable 14.6 points! As with the initial tally, WAPA using more recent polls was most predictive for Sanders and Buttigieg, with the opposite true for Biden. At the same time, Steyer lost 5,383 supporters—and 5.0 points—between initial and final tallies, as did Klobuchar (-2,724; 2.3) and Warren (-1,735, 1.3). For Steyer and Klobuchar, older polls were more predictive, with the opposite true for Warren.

Table 5: Arithmetic difference between Nevada Caucuses CCD % and Nevada Caucuses WAPA

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Mean

Difference

Sanders 22.2 22.0 20.4 19.0 20.9
Biden 0.6 1.2 3.4 4.7 2.5
Warren -3.7 -3.7 -1.7 -1.5 -2.7
Buttigieg 3.4 3.3 1.7 0.6 2.3
Steyer -5.1 -5.1 -7.5 -8.2 -6.5
Klobuchar -2.3 -2.1 -4.1 -5.2 -3.4
Gabbard -1.3 -1.2 -1.4 -1.4 -1.3
Bloomberg -0.1 -0.1 -0.1 0.0 -0.1

CCDs. The same pattern holds for CCDs as for final vote tally.

  • Outperforming their WAPA were Sanders, by an extraordinary 20.9 points, along with Biden and Buttigieg
  • More recent WAPA were more predictive for Sanders and Buttigieg, and less so for Biden
  • Underperforming their WAPA were Steyer, by fully 6.5 points, along with Klobuchar, Warren and Gabbard, with the difference increasing with more recent polls for Steyer and Klobuchar
  • More recent WAPA were more predictive fore Warren; there was no polling period effect for Gabbard.

The bottom line. To evaluate these comparisons globally, I used the sum of the squared differences (“SSE”) between each WAPA value and the results value. Table 6 lists the SSE for each comparison; higher values indicate lower predictive power.

Table 6: SSE of differences between Nevada Caucuses Resuults and Nevada Caucuses WAPA

Polling period Initial Tally Final Tally CCDs
All Polls 124.0 332.1 551.5
Since 1st Debate 118.4 324.1 541.9
Since 5th Debate 80.8 293.7 508.5
Since 7th Debate 64.1 269.7 481.9

WAPA was most predictive of the initial tally, not surprising given that poll respondents are asked which candidate they planned to support upon arriving at the caucus site, and not about second or third choices. WAPA was more predictive of the final raw tally of supporters than of the distribution of CCDs, though neither was especially predictive; I attribute this to Sanders’ astonishing ability both to add supporters during realignment and to acquire at least 15% of the final tally in nearly every precinct caucus.

For each reported measure, WAPA was more predictive the closer the polls were to the Caucuses; this surprised me, given the candidate-specific differences detailed above. One explanation is that including older polls, however low-weighted, masks late polling movement of the kind that occurred to Sanders and Buttigieg.

For now, however, I will continue to report multiple versions of WAPA, if only to see if this pattern holds for later contests.

Now, on to South Carolina!

Until next time…

[1] Percentage of days the poll was being conducted were after the most recent primary or caucuses

Ranking every Marvel Cinematic Universe film

My memory is slightly fuzzy on this point, but I believe I had already heard of the excellent British comedy Coupling the night I happened upon the hysterical Series 4 episode “Nightlines” sometime in late 2004 or early 2005; the show first aired on May 17, 2004. Despite being completely unfamiliar with any of the characters or previous storylines, I have rarely laughed that hard before or since.

And I was hooked, to the point where I have seen all 28 episodes multiple times. In so doing, I learned the name of the man who wrote every episode: Steven Moffat.

A little over five years later, in the late spring of 2010, a friend sent me this short video to watch. This was what finally convinced me to set aside my reticence and watch an episode of Doctor Who; please see here and here to see how THAT turned out.

Among other things, that video marked the advent of Moffat as Doctor Who showrunner, a fitting reward for writing some of the best episodes of the post-2005 revival to that point. It also meant that by the end of 2010, two of my favorite television shows—period—had Moffat’s fingerprints all over them.

This probably made it inevitable, especially given my lifelong obsession with detective fiction, that I—along with my wife Nell—would eventually start watching the television show Moffat co-created and co-wrote with Mark Gatiss,[1] the one which debuted on October 24, 2010, just six months after his tenure as Doctor Who showrunner began:

Sherlock

Though I had seen him act before, in 2013’s Star Trek Into Darkness, Sherlock marked the first time I was aware I was watching an actor named Benedict Cumberbatch.

Flash forward to early January 2020, by which point I had seen every episode of Sherlock, as well as every episode of Coupling and post-revival Doctor Who. Having worked through my obsession with Stranger Things, I was casting about for the next film or television series over which to obsess. I was well aware of the pop culture phenomenon that is the Marvel Cinematic Universe (MCU), but until then I had not been especially interested in watching any of its 23 interconnected films. Curiously, one of our daughters had already seen and loved Guardians of the Galaxy, as well as portions of Avengers: Infinity War, while the other one had seen Captain Marvel on the big screen with one of her best friends. As much as I had enjoyed the Spider-Man trilogy starring Tobey Maguire, though, none of the other characters who seemed to inhabit the MCU—Iron Man, The Incredible Hulk, Captain America and Thor—particularly spoke to me. And that could well be because my primary association with those characters was spending five days as a nine-year-old in early January 1976 staring blankly at Captain America on my Mighty Marvel Bicentennial wall calendar as I recovered from one of the worst flus I have ever had.

Well…there had been one mild exception. When Doctor Strange was released in 2016, starring Cumberbatch in the title role, I was intrigued. Nell and I had loved Cumberbatch in Sherlock, and there was something about his being both a doctor—my Twitter handle is @drnoir33, after all—and a “master of the mystic arts” that felt like a fresh twist on the classic superhero epic battle trope.

Plus, I was really curious about the sparkly golden circles he kept making with his hands.

Which is how I found myself watching—and genuinely enjoying—Doctor Strange roughly six weeks ago. Following a pre-credits fight scene and the opening credits, we meet Doctor Stephen Strange as he prepares for surgery. A short time later, nearing the end of the procedure, he asks a fellow physician to play the “challenge” round in a musical trivia game. After easily identifying “Feels So Good” by Chuck Mangione, along with its correct year of release—1977, not 1978—Strange is asked about all the “useless” knowledge he has.

His flabbergasted response recalls my own immersion in musical esoterica: “Useless? The man charted a top ten hit with a flugelhorn!”

Doctor Strange FunkoPop

As we have been told for years about Pringles, you cannot stop at just one MCU film—especially not when your wife has a massive lifelong crush on Robert Downey, Jr., who portrays Tony Stark/Iron Man in nine MCU films. Subsequent days of film watching culminated with the four of us watching the wholly-satisfying, 3-hour-long Avengers: Endgame on the evening of February 16, 2020; for me, I now only have Spider-Man: Far From Home left to watch. And we are already making plans to see Black Widow when it is released in May 2020.

**********

In two previous posts, I gathered online film rating data to rank…

As I watched the MCU films, I decided to perform a similar analysis of this set of films.[2] Opening a blank Microsoft Excel worksheet, for each film I entered its:

  • Title
  • Date of release (according to the Internet Movie Database, or IMDb)
  • Year of release (ditto)
  • Length in minutes (ditto)
  • Estimated budget (ditto)
  • Gross worldwide earnings (ditto)
  • IMDb score and number of raters
  • Rotten Tomatoes (RT) Tomatometer score (% RT-sanctioned critics deeming film “fresh”), average critic rating (0-10) and number of critics
  • Audience Score (% RT users rating the film 3.5 or higher on 0-5 scale), average user rating and number of user raters

I collected budget and earnings data because I was curious whether, and how much, estimated profit—gross earnings minus budget—was related to perceived quality. Data are current as of 1:30 am EST on February 24, 2020. Analyses were performed using Microsoft Excel (Office Home and Student 2016) and Intercooled Stata 9.2[3].

History of a financial juggernaut.

As Table 1 reveals, the Marvel Cinematic Universe kicked off on May 2, 2008 with the release of Iron Man. Produced for an estimated $140 million, it ultimately earned nearly $585.4 million worldwide; the resulting $445.4 million profit was more than three times what the film cost to make. At the end of the film, in the first MCU post-credits scene, Nick Fury (Samuel L. Jackson) first reveals something called “the Avenger initiative” to Downey’s Stark.

Table 1: MCU Films by release date and financial status

Title Release date Run time (mins.) Estimated budget Gross worldwide earnings Estimated profit Profit/

Budget

Iron Man 5/2/2008 126 $140 million $585,366,247 $445,366,247 3.18
The Incredible Hulk 6/13/2008 112 $150 million $264,770,996 $114,770,996 0.77
Iron Man 2 5/7/2010 124 $200 million $623,933,331 $423,933,331 2.12
Thor 5/6/2011 115 $150 million $449,326,618 $299,326,618 2.00
Captain America: The First Avenger 7/2/2011 124 $140 million $370,569,774 $230,569,774 1.65
Marvel’s The Avengers 5/4/2012 143 $220 million $1,518,812,988 $1,298,812,988 5.90
End of Phase 1
Iron Man 3 5/3/2013 130 $200 million $1,214,811,252 $1,014,811,252 5.07
Thor: The Dark World 11/8/2013 112 $170 million $644,783,140 $474,783,140 2.79
Captain America: The Winter Soldier 4/4/2014 136 $170 million $714,421,503 $544,421,503 3.20
Guardians of the Galaxy 8/1/2014 121 $170 million $772,776,600 $602,776,600 3.55
Avengers: Age of Ultron 5/1/2015 141 $250 million $1,402,805,868 $1,152,805,868 4.61
Ant-Man 7/17/2015 117 $130 million $519,311,965 $389,311,965 2.99
End of Phase 2
Captain America: Civil War 5/6/2016 147 $250 million $1,153,296,293 $903,296,293 3.61
Doctor Strange 11/4/2016 115 $165 million $677,718,395 $512,718,395 3.11
Guardians of the Galaxy Vol. 2 5/5/2017 136 $200 million $863,756,051 $663,756,051 3.32
Spider Man: Homecoming 7/7/2017 133 $175 million $880,166,924 $705,166,924 4.03
Thor: Ragnorak 11/3/2017 130 $180 million $853,977,126 $673,977,126 3.74
Black Panther 2/16/2018 134 $200 million $1,346,913,161 $1,146,913,161 5.73
Avengers: Infinity War 4/27/2018 149 $321 million $2,048,359,754 $1,727,359,754 5.38
Ant-Man and the Wasp 7/6/2018 118 $162 million $622,674,139 $460,674,139 2.84
Captain Marvel 3/8/2019 123 $175 million $1,128,274,794 $953,274,794 5.45
Avengers: Endgame 4/25/2019 181 $356 million $2,797,800,564 $2,441,800,564 6.86
Spider Man: Far From Home 7/2/2019 129 $160 million $1,131,927,996 $971,927,996 6.07
End of Phase 3

Six weeks after Iron Man hit theaters, The Incredible Hulk was released—and while it turned a modest $114.8 million estimated profit, it remains the only MCU film to have a lower estimated profit than estimated budget. Perhaps this is why the third MCU film, Iron Man 2, did not arrive in theaters for nearly two more years; while not as successful as its predecessor, its estimated profit was still more than twice its estimated budget. The same was true of the next two films, which introduced Thor (and Clint Barton/Hawkeye) and Captain America; we had already met Natasha Romanoff/Black Widow in Iron Man 2.

On May 4, 2012, these six “Avengers” would unite in the most successful MCU film to date: Marvel’s The Avengers. This was not only the first film in the franchise to earn more than $1 billion in estimated profit—a staggering 5.9 times its estimated $220 million budget—it is fully 23 minutes longer than the first five films, on average; it also contains my favorite post-credits scene. Avengers provided a highly profitable end to what is now known as “Phase 1,” with the six films combining for more than $2.8 billion in estimated profit.

Phase 2 launched almost exactly one year later with Iron Man 3, the second consecutive MCU film to top $1 billion in estimated profit and have a profit/budget ratio (PBR) of at least 5.0. The next five films, ending with the more explicitly-comedic Ant-Man, all had a PBR of at least 2.79, with Avengers: Age of Ultron becoming the third MCU film to top $1 billion in estimated profit. Overall, the six Phase 2 films earned nearly $4.2 billion in estimated profit, as the franchise steadily increased in popularity. Besides Ant-Man (and, by implication, The Wasp) this Phase also introduced War Machine/James Rhodes, The Falcon/Sam Wilson, Vision/Jarvis, Scarlet Witch/Wanda Maximoff, Rescue/Pepper Potts, Winter Soldier/James “Bucky” Barnes and the Guardians of the Galaxy: Star-Lord/Peter Quill, Rocket Raccoon, Groot, Drax the Destroyer, Gamora and (though not yet an Avenger) Nebula.

Phase 3, the final Phase of what is known collectively as “The Infinity Saga,” began with the release of Captain America: Civil War on May 6, 2016; this film was the longest film to date, at 2 hours, 27 minutes, and it featured the debut of Spider Man. The aforementioned Doctor Strange was released six months later, also introducing Wong, with three films—one introducing Mantis and another introducing Valkyrie, Korg and Miek—following in 2017. The release of Black Panther on February 16, 2018 not only signaled the impending showdown with Thanos in the subsequent Avengers: Infinity War, it also introduced four more Avengers: the titular Black Panther/T’Challa, Okoye, Shuri and M’Baku, bringing the total to 31. Black Panther and Infinity War would become the fourth and fifth MCU films to top $1 billion in estimated profit; the latter’s estimated $1.73 billion in profit easily made it the most profitable film in the franchise to date.

Following an Ant-Man sequel and the introduction of Captain Marvel, the interlocking storylines reached their crescendo on April 25, 2019 with the release of Avengers: Endgame. This latter film, the most profitable of all time at an estimated $2.44 billion—6.9 times its $356 million estimated budget, was just over three hours long, continuing a trend of increasing run times; the previous nine Phase 3 films average 2 hours, 12 minutes in length. Overall, the 11 Phase 3 films accrued $11.16 billion in estimated profit—meaning the average Phase 3 film netted more than $1 billion—bringing the total estimated profit across all 23 MCU films to $18.15 billion, for an average of more than $660 million per film.

As for the sheer length of these films, they combine for 2,996 minutes of run time: 2 days, 1 hour and 56 minutes in total. So, you could knock them off in one weekend-long epic marathon, though I would not recommend it.

Online ratings and increasing public awareness.

Table 2 presents five online ratings and three counts of online raters for the 23 films in the MCU.

Table 2: Ratings Measures for MCU Films

Title IMDb Score

(# Raters)

Tomato-

meter

Mean Tomato-

meter Rating

(# Raters)

RT Audience Score Mean

Audience Rating

(# Raters)

Iron Man 7.9

(898,514)

94 7.7

(278)

91 4.3

(1,082,398)

The Incredible Hulk 6.7

(416,152)

67 6.2

(231)

70 3.7

(739,115)

Iron Man 2 7.0

(686,963)

73 6.5

(297)

71 3.7

(480,400)

Thor 7.0

(711.939)

77 6.7

(284)

76 3.8

(247.469)

Captain America: The First Avenger 6.9

(701,165)

80 6.9

(267)

74 3.8

(188,979)

Marvel’s The Avengers 8.0

(1,218,614)

92 8.1

(384)

91 4.4

(1,135,342)

Iron Man 3 7.2

(721.159)

79 7.0

(318)

78 3.9

(484,684)

Thor: The Dark World 6.9

(565,662)

66 6.2

(271)

76 3.9

(310,425)

Captain America: The Winter Soldier 7.7

(698,659)

90 7.6

(295)

92 4.3

(281,813)

Guardians of the Galaxy 8.0

(996,682)

91 7.8

(322)

92 4.4

(255,076)

Avengers: Age of Ultron 7.3

(700,440)

75 6.8

(360)

83 4.0

(288,171)

Ant-Man 7.3

(533,917)

83 6.9

(321)

86 4.0

(166,462)

Captain America: Civil War 7.8

(621,385)

91 7.7

(406)

89 4.3

(179,582)

Doctor Strange 7.5

(554,767)

89 7.3

(364)

86 4.1

(109,969)

Guardians of the Galaxy Vol. 2 7.6

(624,996)

85 7.3

(403)

87 4.2

(108,403)

Spider Man: Homecoming 7.4

(472,178)

92 7.7

(384)

87 4.2

(107,475)

Thor: Ragnorak 7.9

(534,496)

93 7.6

(409)

87 4.2

(93.959)

Black Panther 7.3

(565,228)

97 8.3

(494)

79 4.1

(88,211)

Avengers: Infinity War 8.5

(748,778)

85 7.6

(455)

91 4.5

(57.790)

Ant-Man and the Wasp 7.1

(277,244)

88 7.0

(417)

76 3.7

(24,169)

Captain Marvel 6.9

(395,538)

78 6.8

(504)

48 2.9

(94,460)

Avengers: Endgame 8.5

(670,991)

94 8.2

(504)

90 4.5

(68,431)

Spider Man: Far From Home 7.5

(264,988)

91 7.5

(427)

95 4.6

(69,222)

 Table 3, meanwhile, summarizes all 14 measures.

 Table 3: Summary MCU Film statistics

Measure Mean

(SD*)

Median Minimum Maximum
Year of Release 2014.7

(3.4)

2015 2008 2019
Length (mins.) 130.3

(15.5)

129 112 181
Estimated Budget $192,782,609

($55,961,802)

$175,000,000 $140,000,000 $356,000,000
Gross Earnings $982,024,151

($576,952,326)

$853,977,126 $264,770,996 $2,797,800,564
Estimated Profit $789,241,543

($52,6038,263)

$663,756,051 $114,770,996 $2,441,800,564
Profit/Budget 3.8

(1.6)

3.5 0.8 6.9
IMDb Score 7.5

(0.5)

7.4 6.7 8.5
# IMDb Raters 629,584.6

(216,222.3)

621,385 277,244

 

1,218,614
Tomatometer 84.8

(8.9)

88 66 97
RT Critic Rating 7.3

(0.6)

7.3 6.2 8.3
# RT Critics 363.7

(80.2)

360 231 504
RT Audience Score 82.4

(10.5)

86 48 92
RT User Rating 4.1

(0.4)

4.1 2.9 4.5
# RT User Raters 289,652.4

(308,813.2)

179,582 24,169 1,135,342

*SD=standard deviation, a measure of how tightly packed values are around the mean: the smaller the value, the tighter the packing. In a normal distribution, 68% of values are within 1 SD, 95% are within 2 SD and 99% are  within 3 SD.

Two conclusions emerge from these data:

  1. As a group, these films are relatively well-regarded
  2. There is minimal variation in how well-regarded these films are.

The median IMDb score for the MCU films is a more-than-respectable 7.4, meaning half the films have a lower score and half have a higher score. Only four films have a score below 7.0: The Incredible Hulk at a good-but-not-great 6.7 and three films at 6.9. The median Tomatometer score was a very-high 88, with a solid average RT Critic rating of 7.3. Only Hulk and Thor: The Dark World have a Tomatometer score less than 70 and an average RT Critic Rating below 6.5. Finally, the median RT Audience Score is an impressive 86 and the median RT User Rating is a very solid 4.1. Only Captain Marvel has an RT audience score below 70 and an average RT User Rating below 3.5, a medicore 48 and 2.9, respectively.

For comparison, the median IMDb Score, Tomatometer, RT Critic Rating, RT Audience Score and RT User Rating values for the 557 films I analyzed in my “guilty pleasures” post are 7.2, 85, 7.1, 76 and 3.5, respectively.

At the other end of the spectrum, meanwhile, Avengers: Infinity War and Avengers: Endgame both have an IMDb score of 8.5, with two other films scoring 8.0.  Fully 10 films have Tomatometer≥90, topped by Black Panther at an eye-popping 97. Black Panther also has the highest RT Critic Rating at 8.3, followed closely by Endgame and The Avengers. Seven films have RT Audience Score≥90, topped by Far From Home at an astonishing 95. Finally, Infinity War, Endgame and Far From Home all have RT User Ratings of 4.5 or 4.6.

As for how little variance there are in these rating measures, all five standard deviations were lower than or (RT User Rating) identical to those for the far more disparate 557 films I analyzed in the earlier post. Broadly speaking, these films are clustered around an appraisal of “good, just shy of great.” Even the (relatively) lower-rated films like Hulk, Dark World and Captain Marvel are far more “meh” than “awful,” while films like Black Panther, The Avengers and Endgame approach “critical darling” status.

The three “number of raters” measures also have relatively low variance. Perhaps because it is the more-established online movie information resource, there are consistently many more IMDb Raters than RT User Raters. At the same time, while none of the 557 films discussed in the earliest post had more than 342 RT Critics, fully 13 MCU films do, topped by the 504 who weighed in on Captain Marvel and Endgame. Curiously, while the number of both IMDb Raters and RT User Raters appears to be lower for more recent films, as one would expect, the number of RT Critics actually seems to increase over time. Correlations (“r”)—a measure ranging from -1.00 to 1.00 of how closely two variables are linearly related to each other[4]—between date of release and each of these three measures confirm this: the former two are negatively correlated (r=-0.47 and -0.78, respectively) with date of release while RT Critics is very highly positively correlated at 0.88.

**********

To assess these films in a more sophisticated way, I used a statistical technique called factor analysis, which groups variables into underlying “dimensions,” or “factors,” used the 14 variables in Table 3. Each variable has a “factor loading” for each factor, essentially its correlation with the underlying dimension. This technique[5] generated three factors accounting for 90% of the total variance in these data, which is remarkably high.

The first factor (accounting for 39% of total variance) is dominated by gross worldwide earnings (0.96), estimated profit (0.96) estimated budget (0.93), run time (0.84) and PBR (0.77); number of RT critics (0.61) and IMDb score (0.56) also load relatively high on this factor. As this dimension mostly combines the cost and profitability of each film with its length, I label it “Epicness.”

The second factor (30%) is dominated by RT audience score (0.91), average RT user rating (0.88), Tomatometer (0.85), RT Critic Rating (0.81) and IMDB Score (0.74). This dimension is clearly “Perceived Quality” (PQ).

The third factor (21%) is dominated by year of release (0.88), number of RT audience raters (-0.84), number of RT critics (-0.74) and number of IMDB raters (0.72): precisely the same pattern outlined above. This dimension is effectively “Recency;” I do not dwell on it below, echoing the “guilty pleasures” post.

Table 4: How MCU Films Compare on Three “Ratings” Dimensions

Title

Epicness

Perceived Quality Recency
Iron Man -0.66 1.19 -1.97
The Incredible Hulk -0.81 -1.55 -1.25
Iron Man 2 -0.12 -1.27 -1.01
Thor -0.80 -0.61 -0.54
Captain America: The First Avenger -0.96 -0.55 -0.39
Marvel’s The Avengers 1.05 1.01 -1.64
Iron Man 3 0.68 -0.73 -0.92
Thor: The Dark World -0.35 -1.30 -0.48
Captain America: The Winter Soldier -0.64 0.97

-0.30

Guardians of the Galaxy -0.54 1.24 -0.48
Avengers: Age of Ultron 0.96 -0.76 -0.41
Ant-Man -1.05 0.15 0.35
Captain America: Civil War 0.33 0.62 0.19
Doctor Strange -0.67 0.44 0.55
Guardians of the Galaxy Vol. 2 -0.36 0.30 0.73
Spider Man: Homecoming -0.42 M 0.64 0.80
Thor: Ragnorak -0.48 0.76 0.81
Black Panther 0.47 0.49 1.06
Avengers: Infinity War 1.87 0.32 M -0.03 M
Ant-Man and the Wasp -0.74 -0.35 1.68
Captain Marvel 0.68 -2.47 1.50
Avengers: Endgame 2.98 0.47 0.33
Spider Man: Far From Home -0.66 1.19 -1.97

Table 4 reveals how many SD above or below the mean (set to 0) all 23 films are on these three dimensions.[6] Values≥1.0 are boldfaced, and values≤-1.0 are italicized; median value is marked with an “M.”

When reading these values, keep in mind that each of these factors is as “disentangled” from the other two as possible, though Epicness and PQ still overlap to some extent. This is why, for example, Infinity War and Endgame have by far the highest “Epicness” scores—they are the longest films with the highest budgets earning the most money—but do not have as high PQ scores despite their generally high ratings: they are far more “epic” than they are “high quality” according to these data. And it is why Guardians and Iron Man top these films on PQ—they are the highest-rated films which, while very profitable, were not quite on the scale of the final two Avengers films; the well-received Captain America: The Winter Soldier falls into this category as well. Somewhere in between are epic, but not well-regarded films like Ultron and less-epic, but relatively well-regarded films like Ant-Man and Doctor Strange.

The only film, meanwhile, with value≥1.0 on both measures is The Avengers, while the only film to have positive values on all three measures is Civil War.

At the other end of the spectrum, not surprisingly, are films like Hulk, Dark World, and Iron Man 2 that made far less money and are relatively lower-rated, as well as the anomalous Captain Marvel, which turned a tidy profit despite the lowest PQ score by far. In fact, every film between Iron Man and Avengers has negative values for all three measures, as does Dark World.

Summary. For those new to the MCU, these data suggest starting at the beginning, with Iron Man then jumping ahead to The Avengers; you do not miss much along the way, with the mild exception of First Avenger, which introduces key characters and plot points. Watch Winter Soldier and Guardians next, then Civil War. I personally would watch Ant-Man, Doctor Strange and Black Panther after that, if only because each is interesting in its own right and, like First Avenger, relay key characters and plot points. And then you can conclude with Infinity War and Endgame, bearing in mind their combined run time is 5 hours and 30 minutes.

Or, you can choose your own MCU adventure, which these data strongly suggest you would enjoy.

Until next time…

[1] I strongly recommend Gatiss’ three-part series on the history of horror films. Part 1 may be found here.

[2] Only to learn Leonard Maltin stopped publishing his annual Movie Guide in 2015. ata Statistical Software: Release 9. College Station, TX: StataCorp LP.

[4] Essentially, a positive correlation means that as one variable increases, the other one does as well, while a negative correlation means that as one variable increases, the other one decreases.

[5] Principal factors, with an orthogonal varimax rotation, forced to three factors.

[6] Using the “Predict” command—regression scoring method—in Stata. In essence, it converts each variable to a “z-score” (mean=0, SD=1), recalculates the factor loadings, then sums each value weighted by the factor loadings.

2020 Nevada Caucuses: Final Polling Update

At 10 am Nevada time (1 pm EST) on February 22, 2020, Nevadans who did not already vote between February 15 and 18 by submitting a ranking of their three top choices to be the 2020 Democratic nominee will gather in nearly 2,100 meeting places to support their preferred candidate(s). While about 84,000 Nevadans caucused for Democrats in total in 2016, the number of Nevadans who voted early in 2020 is estimated to be at least 70,000.

This post presents four variations on my final pre-Caucuses WAPA, calculated using the four different timeframes detailed below. To learn how I calculate candidate WAPA (weighted-adjusted polling averages), please see here. Polls conducted partially or entirely after the Iowa Caucuses, but before the New Hampshire Primary, are weighted up to 1.33 to 2.00 times higher than polls conducted before the Iowa Caucuses. Polls conducted partially or entirely after the New Hampshire Primary are weighted 2.33 to 3.00 times higher than polls conducted before the New Hampshire Primary.

I have never been to Nevada, so here is the February 2020 lighthouse photograph in my Down East 2020 Maine Lighthouses wall calendar.

Feb 2020 lighthouse

**********

As of midnight EST on February 22, 2020, here is the breakdown of publicly-available Nevada Caucuses polls.

  • 22 conducted since January 1, 2019
  • 19 conducted since the 1st Democratic debate on June 26, 2019
  • 10 conducted since the 5th Democratic debate on November 19, 2019
  • 7 conducted since the 7th Democratic debate on January 14, 2020

First, however, here are some words of caution.

1. Very few high-quality polls

Only 22 polls of the Nevada Caucuses have been conducted since January 1, 2019, considerably fewer than the 59 polls of the Iowa Caucuses and the 72 polls of the New Hampshire Primary conducted over the same period of time. Overall, the 15 pollsters operating in Nevada, as a group, compare favorably to the 19 pollsters who operated in Iowa and the 21 who operated in New Hampshire: all three sets of pollsters averaged a B/B-, using FiveThirtyEight.com pollster ratings. However, only one of the seven polls conducted since the January 14, 2020 Democratic debate was conducted by a pollster who do have either a B/C (three) or a C+ (three) rating: an Emerson College poll (A-) conducted February 19-20, 2020. This poll is weighted 2.57; no other poll is weighted higher than 1.76.

The combination of many fewer polls, a dearth of recent high-quality polls and the fact polling in Nevada is extremely challenging in the best of circumstances makes these final Nevada Caucuses WAPA even more “wobbly” than those for the unpredictable Iowa Caucuses.

2. Reporting snafus 2.0?

Recall that in a caucus setting, voters literally declare their preferences in a public setting. If, after an initial tabulation, a candidate does not have the support of at least 15% of that site’s caucus attendees, supporters of that candidate either join a group that does (i.e., is “viable”), the “uncommitted” group or call it a day. This realigning continues until only viable candidates remain, at which time a complex formula is used to calculate “state delegate equivalents,” or SDEs. Prior to 2020, only the percentage of SDEs won were reported.

However, as was the case in Iowa, this year the Nevada Democratic Party plans to report three sets of results:

  • The initial statewide tally for each candidate
  • The post-viability tally for each candidate
  • SDE’s for each candidate

Having to report all three tallies in Iowa proved so difficult it ultimately cost the state’s Democratic Party Chair, Troy Price, his job. The Nevada Democratic Party has reassured us it is doing everything it can to report the results of its caucuses in a timely manner, though accuracy should always be prioritized over speed. There is a vast difference between conducting an election and reporting the results of that election; the former should take precedence over the latter, whatever the discomfiture level of journalists waiting to report results.

With those two caveats, Table 1 presents the final pre-Caucuses WAPA for the eight remaining candidates for the 2020 Democratic presidential nomination; even though former New York City Mayor Michael Bloomberg is not actively campaigning in Nevada, he is included as Nevadans could theoretically caucus for him.

Table 1: Final Nevada Caucuses WAPA for declared 2020 Democratic presidential nomination candidates

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate
Sanders 24.6 24.8 26.4 27.8
Biden 19.6 19.0 16.8 15.5
Warren 13.4 13.4 11.4 11.2
Buttigieg 10.9 11.0 12.6 13.7
Steyer 9.8 9.8 12.2 12.9
Klobuchar 6.5 6.3 8.3 9.4
Gabbard 1.4 1.3 1.5 1.5
Bloomberg 0.1 0.1 0.1 0.0
DK/Other 13.7 14.4 10.6 7.9

United States Senator (“Senator”) from Vermont Bernie Sanders would appear to have a significant edge—and momentum—heading into the final day of caucusing in Nevada. Increasingly falling behind Sanders are five candidates bunched relatively close together: former Vice President Joe Biden; Massachusetts Senator Elizabeth Warren; former South Bend, Indiana Mayor Pete Buttigieg; billionaire activist Tom Steyer and Minnesota Senator Amy Klobuchar. However, while Buttigieg, Steyer and Klobuchar appear to have some momentum heading into Saturday, Biden and Warren are fading somewhat, though we have no idea how Warren’s dominant February 19 debate performance will impact her final results. After a similarly-strong performance, Klobuchar outperformed her final New Hampshire WAPA by an average of 11.5 percentage points (“points”); were Warren to jump even one-third that amount—around four points—she could easily finish in 2nd place.

The bottom line.

As with the earlier Iowa Caucuses, anybody who thinks they know what will happen in the 2020 Nevada Democratic Caucuses has absolutely no idea what will happen in the 2020 Nevada Democratic Caucuses. Polling is very difficult to do in a multi-candidate race, with caucuses adding the additional wrinkle of unknowable “backup” choices or even who will participate. Combine this with a noticeable lack of recent high-quality polling, and we have something more akin to a gut-level hunch than a scientific projection.

It may very well be Sanders is the heavy favorite to win the Nevada Caucuses with literally every other candidate struggling to reach the 15% viability threshold, but I would not bet anything remotely of value on it.

We shall see.

Until next time…

2020 New Hampshire Primary: How did my polling averages fare?

Given the extremely volatile polling for the 2020 Democratic presidential nomination following the start of primary and caucus voting, I will not provide global monthly updates for the next few months. Instead, I will focus on the first handful of primaries and caucuses: Iowa on February 3, New Hampshire on February 11, Nevada on February 22, South Carolina on February 20, the 14 Super Tuesday contests on March 3, and so forth.

Also: I now weight higher polls conducted partially (1.33 or 1.67) or entirely (2.00) after February 3, 2020, than polls conducted entirely before February 4, 2020. I similarly weight higher polls conducted partially (2.33 or 2.67) or entirely (3.00) after Feb 11, 2020.

Unlike the molasses-slow pace it took for results from the 2020 Iowa Caucuses to be released—ultimately leading to the resignation of Iowa Democratic Party Chair Troy Price—95+% of the results from the 2020 New Hampshire Primary were tabulated and released within five hours of the final poll closings at 8 pm EST on February 11, 2020.

This improvement is undoubtedly due to the two “sharing size” bags of M&M’s—one plain, one peanut—I purchased at our local CVS after dropping our eldest daughter at her swim team workout. These candies have long been a staple of our election night watch parties, yet I neglected to buy some for the Iowa Caucuses the previous week.

Scan0046

Earlier that day, meanwhile, I had published my final New Hampshire Primary WAPA (weighted-adjusted polling average) for the 11 then-declared Democratic presidential candidates, calculated five different ways (Table 1):

  • 71 since January 1, 2019
  • 57 since the 1st Democratic debate on June 26, 2019
  • 35 since the 5th Democratic debate on November 19, 2019
  • 27 since the 7th Democratic debate on January 14, 2020
  • 15 since the Iowa Caucuses on February 3, 2020
    • 14 starting February 4 or later
    • 1 (Monmouth University) on February 3

Table 1: Final New Hampshire Primary WAPA for declared 2020 Democratic presidential nomination candidates

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Since

Iowa

Sanders 23.0 23.1 24.8 25.5 26.4
Buttigieg 16.7 17.0 18.8 19.4 21.4
Biden 15.8 15.3 14.0 13.0 12.2
Warren 14.3 14.7 12.9 12.6 13.1
Klobuchar 7.0 7.2 8.6 9.3 9.7
Gabbard 3.4 3.5 3.7 3.5 3.2
Yang 3.2 3.3 3.6 3.4 3.2
Steyer 2.5 2.6 2.9 2.8 2.6
Bennet 0.4 0.5 0.5 0.5 0.6
Bloomberg 0.3 0.3 0.4 0.3 0.1
Patrick 0.3 0.3 0.4 0.4 0.4
DK/Other 13.1 12.2 9.6 9.2 7.1

Based solely on these numbers, one could reasonably draw the following conclusions:

  • United States Senator (“Senator”) from Vermont Bernie Sanders; former South Bend, Indiana Mayor Pete Buttigieg; and Minnesota Senator Amy Klobuchar were rising in the polls heading into the Iowa Caucuses.
  • Former Vice President Joe Biden and Massachusetts Senator Elizabeth Warren were declining in the polls.
  • No other candidate was moving in the polls one way or the other.

Comparing WAPA to results. Table 2 lists the results of the 2020 New Hampshire Democratic Primary as of 2 am EST on February 13, 2020:

Table 2: Percentage of vote received in 2020 New Hampshire Democratic Primary

Candidate % Vote
Sanders 25.8
Buttigieg 24.5
Klobuchar 19.9
Warren 9.3
Biden 8.4
Steyer 3.6
Gabbard 3.3
Yang 2.8
Write-in votes (including Bloomberg) 1.4
Patrick 0.4
Bennet 0.3
All others 0.4

Table 3 lists the arithmetic differences between each candidate’s final Iowa Caucuses WAPA and each of the three reported measures; positive values indicate better performance in the Caucuses than in the polls.

Table 3: Arithmetic difference between Vote % and WAPA, 2020 New Hampshire Democratic Primary

Candidate All

Polls

Since 1st Debate Since 5th Debate Since 7th Debate Since

 Iowa

Mean

Differnce

Sanders 2.8 2.7 1.0 0.3 -0.6 1.2
Buttigieg 7.8 7.5 5.7 5.1 3.1 5.8
Biden -7.4 -6.9 -5.6 -4.6 -3.8 -5.7
Warren -5.1 -5.5 -3.7 -3.4 -3.9 -4.3
Klobuchar 12.9 12.7 11.3 10.6 10.2 11.5
Gabbard -0.1 -0.2 -0.4 -0.2 0.1 -0.2
Yang -0.4 -0.5 -0.8 -0.6 -0.4 -0.5
Steyer 1.1 1.0 0.7 0.8 1.0 0.9
Bennet -0.1 -0.2 -0.2 -0.2 -0.3 -0.2
Bloomberg 0.4 0.4 0.3 0.4 0.6 0.4
Patrick 0.1 0.1 0.0 0.0 0.0 0.0

Two candidates—Buttigieg and Klobuchar—strongly outperformed their final New Hampshire Primary WAPA, by an average of 5.8 and 11.5 percentage points (“points”), respectively. The polling momentum they appeared to have coming out of the Iowa Caucuses was apparent in the final results, especially given that the polls conducted in the following week were the least inaccurate.

Sanders narrowly edged Buttigieg, 25.8% to 24.5%, to win the 2020 New Hampshire Democratic Primary, finishing an average 1.2 points higher than his WAPA; again WAPA was more accurate the closer in time it was calculated. Businessman Tom Steyer also finished nearly one point better in the voting (3.6%) than his average WAPA, which differed little in accuracy by time of calculation.

By contrast, Biden (-5.7 points) and Warren (-4.3 points) strongly underperformed their WAPA, with the more recent estimates again the least inaccurate.

One immediate consequence of these results is that three candidates, Colorado Senator Michael Bennet, former Massachusetts Governor Deval Patrick and entrepreneur Andrew Yang, ended their presidential campaigns. As with the 15 former candidates who preceded them, all three men did so with grace and class. One other candidate, United States House of Representatives member from Hawaii Tulsi Gabbard, finished 7th in New Hampshire, barely registered in Iowa and is not close to making any future debates, so her status in this race is somewhat shaky.

Bottom line. To evaluate these comparisons globally, I calculated two difference measures for each of the five WAPA, excluding “DK/Other” (Table 4):

  1. Means of the absolute value of each candidate’s value in Table 3
  2. Sums of the squared differences (“SSE”) between each of the five WAPA value and the results value.

Table 4: Global differences between WAPA and results, 2020 New Hampshire Democratic Primary

Polling period Mean AV Difference SSE
All Polls 3.5 317.4
Since 1st Debate only 3.4 304.2
Since 5th Debate only 2.7 207.7
Since 7th Debate only 2.4 172.4
Since Iowa Caucuses only 2.3 145.3

All five versions of WAPA were quite accurate, despite sharp late movement by Biden, Buttigieg, Klobuchar and Warren, missing by between 2.3 and 3.5 points in either direction, on average. Not surprisingly, the older the collection of polls used to calculate WAPA, the higher the average “miss.”

As with the Iowa Caucuses, meanwhile, and despite newer polls greatly outweighing older polls: the older the set of polls used to calculate New Hampshire Democratic Primary WAPA values, the less predictive they were of the actual results. And, again, the older-polls WAPA may shift too slowly to capture significant late movement.

Given this consistency over the first two contests, I will continue to use this template to assess WAPA.

Now, on to the Nevada Caucuses on February 22, 2020!

Until next time…

2020 New Hampshire Primary: Final Polling Update

[This poll was updated at 4:15 EST on February 11, 2020 to account for a Change Research poll (C) conducted February 8-9, 2020.]

At just after midnight EST on February 11, 2020, a total of 27 Democratic voters gathered in the small New Hampshire hamlets of Dixville Notch, Hart’s Location and Millsfield to cast the first votes in the 2020 New Hampshire Presidential Primary. Here are the tallies from those early votes (which, while fun to watch, have historically shown little relationship to the way the statewide electorate votes); I jotted them down as they were being broadcast live on MSNBC:

Table 1: 2020 New Hampshire Presidential Primary votes from three towns voting just after midnight

Candidate Dixville Notch Hart’s Location Millsfield Total
Klobuchar 0 6 2 8
Sanders 1 2 1 4
Warren 0 4 0 4
Yang 0 3 0 3
Biden 0 1 1 2
Buttigieg 1 0 1 2
Bloomberg 2 0 0 2
Steyer 0 1 0 1
Gabbard 0 1 0 1
TOTAL 4 18 5 27

The perhaps-surprising winner in these early returns is United States Senator (“Senator”) from Minnesota Amy Klobuchar, who had a very well-received debate performance in Manchester, New Hampshire on February 7; as the data in Table 2 suggest, she may have some momentum from that performance. Also, the name of former New York City Mayor Michael Bloomberg does not appear on the either the Democratic or Republican New Hampshire Primary ballot; these votes, as well as the only Republican vote cast in Dixville Notch, are write-in votes.[1]

To learn how I calculate candidate WAPA (weighted-adjusted polling average), please see here. I weight polls conducted partially (1.33 or 1.67) or wholly (2.00) after the Iowa Caucuses higher than prior polls.

Here is a photograph of a now-defunct ice cream parlor in scenic Portsmouth, New Hampshire in early September 2005; I was never the owner of this establishment:

Scan0046

**********

As of 2 a.m. EST on February 11, 2020, here is a breakdown of publicly-available New Hampshire Primary polls:

  • 72 since January 1, 2019
  • 58 since the 1st Democratic debate on June 26, 2019
  • 36 since the 5th Democratic debate on November 19, 2019
  • 28 since the 7th Democratic debate on January 14, 2020
  • 16 since the Iowa Caucuses on February 3, 2020
    • 15 starting February 4 or later
    • 1 (Monmouth University) on February 3

The final two tracking polls by Emerson College and the University of New Hampshire overlapped in time with each organization’s prior tracking polls, so I only used the most recent ones.

Table 2: Final New Hampshire WAPA for declared 2020 Democratic presidential nomination candidates

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Since

Iowa

Sanders 23.0 23.1 25.0 25.7 26.6
Buttigieg 16.7 17.0 18.8 19.4 21.4
Biden 15.7 15.2 13.9 12.9 12.1
Warren 14.0 14.2 12.8 12.5 12.7
Klobuchar 7.0 7.2 8.5 9.3 9.6
Gabbard 3.5 3.6 3.8 3.6 3.3
Yang 3.3 3.4 3.6 3.5 3.3
Steyer 2.5 2.6 2.9 2.8 2.6
Bennet 0.4 0.4 0.5 0.5 0.5
Bloomberg 0.3 0.3 0.4 0.2 0.1
Patrick 0.3 0.3 0.4 0.4 0.5
DK/Other 13.4 12.6 9.4 9.2 7.3

Besides Klouchar, the top two finishers in the Iowa Caucuses—Vermont Senator Bernie Sanders and former South Bend, Indiana Mayor Pete Buttigieg—also appear to have momentum going into New Hampshire. The likelihood is that Sanders will win the primary—though not by anywhere close to his 2016 winning margin of 60% to 38% over former Secretary of State Hillary Clinton—perhaps followed by Buttigieg.

By contrast, former Vice President Joe Biden and, to a lesser extent, Massachusetts Senator Elizabeth Warren appear to be fading somewhat. United States of House of Representatives member Tulsi Gabbard of Hawaii, entrepreneur Andrew Yang and businessman Tom Steyer remain clustered around 2.5-3.7 percentage points, while Colorado Senator Michael Bennet and former Massachusetts Governor Deval Patrick appear to be non-factors at this point.

We shall see.

Until next time…

[1] President Donald Trump received 15 votes in Hart’s Location and 16 votes in Millsfield, while former Massachusetts Governor William Weld received 4 votes and 1 vote, respectively. Mary Maxwell received 1 vote in Hart’s Location.

2020 Iowa Caucuses: How did my polling averages fare?

Given the extremely volatile polling for the 2020 Democratic presidential nomination following the conclusion of the Iowa Caucuses, I will not provide global monthly updates for next few months. Instead, I will focus on the first handful of primaries and caucuses: Iowa on February 3, New Hampshire on February 11, Nevada on February 22, South Carolina on February 20, the 14 Super Tuesday contests on March 3, and so forth.

Also: I now weight polls conducted partially after February 3, 2020 either 1.333 or 1.667 times higher, and polls conducted entirely after February two times higher, than polls conducted entirely before February 4, 2020.

On the night of February 3, 2020, I was sitting on my usual spot on our sofa, watching MSNBC and anticipating returns from that day’s Iowa Caucuses.

Iowa Visitor Center Sep 1990

Earlier that day, I had published my final WAPA (weighted-adjusted polling average) for the 11 declared Democratic presidential candidates, calculated four different ways (Table 1):

  • Using all 58 polls conducted since January 1, 2019
  • Using only the 45 polls released since the 1st Democratic debate on June 26, 2019
  • Using only the 21 polls released since the 5th Democratic debate on November 19, 2019
  • Using only the 15 polls released since the 7th Democratic debate on January 14, 2020

Table 1: Final Iowa Caucuses WAPA for declared 2020 Democratic presidential nomination candidates

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate
Biden 19.9 19.8 20.1 20.3
Sanders 18.4 18.8 21.0 22.7
Warren 17.1 18.1 15.6 15.6
Buttigieg 15.9 16.8 16.7 16.7
Klobuchar 6.9 7.3 9.1 9.7
Yang 3.0 3.2 3.6 3.9
Steyer 2.8 3.1 3.1 3.5
Gabbard 1.5 1.6 1.5 1.6
Bloomberg 0.4 0.4 0.6 0.5
Bennet 0.3 0.3 0.2 0.3
Patrick 0.0 0.0 0.0 0.1
DK/Other 13.8 10.6 8.5 5.2

Based solely on these numbers, one would reasonably draw the following conclusions:

  • United States Senator (“Senator”) from Vermont Bernie Sanders and Minnesota Senator Amy Klobuchar were rising in the polls heading into the Iowa Caucuses, as to a lesser extent were entrepreneur Andrew Yang and businessman Tom Steyer.
  • Massachusetts Senator Elizabeth Warren was declining in the polls.
  • No other candidate was moving in the polls one way or the other.

By 11:37 pm EST, however, I had grown tired of waiting for results other than successive waves of entrance polls, so I tweeted the following:

RIP, Iowa Caucuses (1972-2020)

I have defended their idiosyncrasies for decades, believing the retail aspects of campaigning there outweighed the low-turnout mischegoss of the process.

 No more.

 This is ridiculous.

 #IowaCaucuses #iowacaucus2020

I will not relitigate here the myriad problems the Iowa Democratic Party had with tabulating, validating and releasing three distinct measures:

  1. Initial headcount of support for each Democratic candidate (“Initial tally”)
  2. Post-realignment headcount of support for each Democratic candidate (“Final tally”)
  3. Allocation of “state delegate equivalents,” or SDE’s, the only measure ever previously reported

Moreover, my annoyance has abated since Monday night, primarily because I suspect these vote-reporting snafus revealed that the byzantine process of converting persons standing in rooms, then possibly standing in different parts of the room, into SDE’s has always been “riddled with errors and inconsistencies,” to quote a recent New York Times headline. And if this marks the beginning of the end of using caucuses to allocate delegates to each party’s nominating conventions, so be it; they are undemocratic, exclusionary and overly complex.

As for which states “should” come first in future presidential nominating processes, I am currently agnostic.

Three days later, we finally have near-final results from the Iowa Caucuses (Table 2):

Table 2: Near-final Iowa Democratic Caucuses results, February 3, 2020

Candidate Initial Tally Final Tally SDE’s
Biden 15.0 13.7 15.8
Sanders 24.8 26.6 26.1
Warren 18.4 20.2 18.0
Buttigieg 21.3 25.0 26.2
Klobuchar 12.7 12.3 12.3
Yang 5.0 1.0 1.0
Steyer 1.7 0.2 0.3
Gabbard 0.2 0.0 0.0
Bloomberg 0.1 0.0 0.0
Bennet 0.1 0.0 0.0
Patrick 0.0 0.0 0.0
Uncommitted 0.6 0.1 0.2

The following three tables list the arithmetic differences between each candidate’s final Iowa Caucuses WAPA and each of the three reported measures; positive values indicate better performance in the Caucuses than in the polls.

Table 3: Arithmetic difference between Initial Iowa Caucuses % of vote and Iowa Caucuses WAPA

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Mean

Difference

Biden -4.9 -4.8 -5.1 -5.3 -5.0
Sanders 6.4 6.0 3.8 2.1 4.6
Warren 1.3 0.3 2.8 2.8 1.8
Buttigieg 5.4 4.5 4.6 4.6 4.8
Klobuchar 5.8 5.4 3.6 3.0 4.5
Yang 2.0 1.8 1.4 1.1 1.6
Steyer -1.1 -1.4 -1.4 -1.8 -1.4
Gabbard -1.3 -1.4 -1.3 -1.4 -1.4
Bloomberg -0.3 -0.3 -0.5 -0.4 -0.4
Bennet -0.2 -0.2 -0.1 -0.2 -0.2
Patrick 0.0 0.0 0.0 -0.1 0.0
DK/Other -13.2 -10.0 -7.9 -4.6 -8.9

Initial tally. If the Iowa Caucuses were instead the Iowa Primary, this would have been the only vote reported. On this measure Sanders, Klobuchar and former South Bend, IN Mayor Pete Buttigieg averaged 4.5-4.8 percentage points (“points”) higher in the initial tally than in their WAPA. And the closer in time the polls were to the Iowa Caucuses, the more “accurate” the WAPA.

Warren (+1.8 points) and Yang (+1.6) also overperformed their WAPA in the initial tally, albeit by smaller margins. And for Warren, older polls were more predictive than recent polls.

By contrast, former Vice President Joe Biden did an average of 5.0 points worse in the initial Iowa Caucuses tally than his WAPA. Steyer and United House of Representatives Member from Hawaii Tulsi Gabbard (-1.4 each) also performed somewhat worse than their WAPA.

Table 4: Arithmetic difference between Final Iowa Caucuses % of vote and Iowa Caucuses WAPA

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Mean

Difference

Biden -6.2 -6.1 -6.4 -6.6 -6.3
Sanders 8.2 7.8 5.6 3.9 6.4
Warren 3.1 2.1 4.6 4.6 3.6
Buttigieg 9.1 8.2 8.3 8.3 8.5
Klobuchar 5.4 5.0 3.2 2.6 4.1
Yang -2.0 -2.2 -2.6 -2.9 -2.4
Steyer -2.6 -2.9 -2.9 -3.3 -2.9
Gabbard -1.5 -1.6 -1.5 -1.6 -1.6
Bloomberg -0.4 -0.4 -0.6 -0.5 -0.5
Bennet -0.3 -0.3 -0.2 -0.3 -0.3
Patrick 0.0 0.0 0.0 -0.1 0.0
DK/Other -13.7 -10.5 -8.4 -5.1 -9.4

Final tally. Only three candidates improved their vote totals after supporters of non-viable candidates shifted to a viable candidate (15% of attendees at a precinct caucus):

  • Buttigieg (+5,638 supporters; +3.7 points)
  • Warren (+2,238; +1.8)
  • Sanders (+2,155; +1.8)

These three candidates, as well as Klobuchar (-1,288; -0.4), performed better in the final tally than their WAPA, on average. As with the initial tally, WAPA using more recent polls was most predictive for Sanders, Buttigieg and Klobuchar, while WAPA using older polls was most predictive for Warren.

Biden, on the other hand, lost 2,693 supporters and dropped 1.3 points between the initial and final tallies; Yang and Steyer also lost considerable support between the initial and final tallies. For all three candidates, WAPA using earlier polls was most predictive.

Table 5: Arithmetic difference between Iowa Caucuses SDE % and Iowa Caucuses WAPA

Candidate All Polls Since 1st Debate Since 5th Debate Since 7th Debate Mean

Difference

Biden -4.1 -4.0 -4.3 -4.5 -4.2
Sanders 7.7 7.3 5.1 3.4 5.9
Warren 0.9 -0.1 2.4 2.4 1.4
Buttigieg 10.3 9.4 9.5 9.5 9.7
Klobuchar 5.4 5.0 3.2 2.6 4.1
Yang -2.0 -2.2 -2.6 -2.9 -2.4
Steyer -2.5 -2.8 -2.8 -3.2 -2.8
Gabbard -1.5 -1.6 -1.5 -1.6 -1.6
Bloomberg -0.4 -0.4 -0.6 -0.5 -0.5
Bennet -0.3 -0.3 -0.2 -0.3 -0.3
Patrick 0.0 0.0 0.0 -0.1 0.0
DK/Other -13.6 -10.4 -8.3 -5.0 -9.3

SDEs. The same pattern holds for SDEs as for final vote tally, with one minor modification.

  • Buttigieg, Sanders and Klobuchar outperformed their WAPA, with the difference decreasing with more recent polls
  • Warren outperformed her WAPA, with the difference increasing with more recent polls
  • Biden, Steyer and Yang underperformed their WAPA, with the difference increasing with more recent polls.

The bottom line. To evaluate these comparisons globally, I used the sum of the squared differences (“SSE”) between each WAPA value and the results value. Excluding “DK/Other,” Table 6 lists the SSE for each comparison; higher values indicate lower predictive power.

Polling period Initial Tally Final Tally SDEs
All Polls 136.5 240.5 224.9
Since 1st Debate 115.8 210.8 198.2
Since 5th Debate 88.3 190.4 168.0
Since 7th Debate 77.1 177.8 156.1

WAPA was most predictive of the initial tally, not surprising given that poll respondents are asked which candidate they planned to support upon arriving at the caucus site, and not about second or third choices. WAPA was also slightly more predictive of the distribution of SDEs than of the final raw tally of supporters, though neither was especially predictive.

For each reported measure, WAPA was more predictive the closer the polls were to the Caucuses; I will admit this rather surprised me, given the candidate-specific differences detailed above. One explanation is that including older polls, however low-weighted, masks late polling movement of the kind that occurred to Sanders, Buttigieg and Klobuchar.

For now, however, I will continue to report multiple versions of WAPA, if only to see if this pattern holds for later contests.

Now, on to New Hampshire!

Until next time…