Dispatches from Brookline: Home Schooling and Social Distancing VI

I have described elsewhere how my wife Nell, our two daughters—one in 4th grade and one in 6th grade—and I were already coping with social distancing and the closure of the public schools in Brookline, Massachusetts until at least April 7, 2020. Besides staying inside as much as possible, we converted our dining room into a functioning classroom complete with workbooks, flip charts and a very popular white board.

**********

When I came downstairs to eat what I continue to call breakfast—despite it being closer to 2:30 pm than, say, 8:30 am—this is what was in the “classroom.”

March 26

Nell appears to have discontinued the “Word of the day” for now. She also left the title of my afternoon classes to our younger daughter’s discretion. However, the press of the latter’s still-active social life kept her from formulating a suitable name, so I stepped in to fill the void.

And, in fact, when the girls and I convened, closer to 3 pm than 2:30 pm, we began by reading aloud from the Constitution of the United States:

  • Article I, Section 7, Paragraph 1
  • Article I, Section 8

US Constitution–Congress Roles

The rest of the lesson may be found here: March 26

The traditional processes by which the United States House of Representatives (“House”) passes legislation was met with a metaphorical yawn, but the workings of the United States Senate (“Senate”) generated a bit more enthusiasm. Our younger daughter, in particular, was quite interested in the twists and turns of getting the Affordable Care and Patient Accountability Act—better known as Obamacare—passed, and she was riveted by the pivotal role Arizona Senator John McCain played in saving it. I did my best to act out McCain’s dramatic “thumbs down” vote.

Nell and I are continuing to learn how best to structure what, when and how we teach our daughters—when they are not working and learning on their own. Seeing how fragile our younger daughter—who has attention deficit disorder and a not-yet-formally-diagnosed learning disability—is by 5 pm, I mixed things up a bit.

I also wanted to avoid snapping at them for the third time this week.

Rather than discuss American politics for an hour, have an hour-long break, then reconvene for another hour-long session on applied math, I divided my discussion of the House and Senate into two parts: roles and elections. The break was only 20 minutes long, and we were finished for the day by 5 pm.

As you see, I spent some time discussing gerrymandering. Our older daughter was appalled at my drawing of a salamander—calling it a “giant worm”—and my rendition of the Commonwealth of Massachusetts. She took it upon herself to fix the latter, adding her own personal touches.

Gerrymandered

I am pleased to report this was one of our best classes thus far—and that includes both halves.

**********

Even with the altered routine, however, our daughters began sniping at each other as they ate their dinners and watched some television. The proximate cause was a tussle over who would hold the remote control; our older daughter usually does, but tonight her sister really wanted it. As a result, our older daughter called her sister a “brat,” something she has been admonished many times in the past for doing. In return, our younger daughter used the parental trick of calling an obstreperous child her first, middle and last name—but she used a shrill and piercing tone of voice.

They were sitting just outside of my home office—a converted sun room which Nell wistfully calls “the nicest room in the apartment”—when I heard the outraged cry of “Blanky Blank Berger!” Not in the best frame of mind, I came out of my office to express my displeasure at the younger daughter’s tone and to make clear she is not the parent, Nell and I are.

And, wow, did I lay it on thick. I reminded them in my firmest and harshest Daddy voice how we were in this for the long haul, and we needed to do this all together, and I do not even remember what all else. Younger daughter was now crying—but mostly because of the injustice that I had not tumbled to the fact her sister had called her a brat. Once that penetrated my skull, though, I reprimanded our older daughter. Walking in from the kitchen, Nell reinforced my disapproval. When I suggested the older daughter had earned a consequence, her mother suggested loss of the ChromeBook for the rest of the evening. However, once the defendant correctly pointed out the usual consequence for calling her sister a brat is to cough up five dollars to that sister, Nell realized she could not arbitrarily change the rules like that; a few minutes later, our younger daughter had a five-dollar bill sitting on the table in front of her. And the entire episode, which had lasted barely ten minutes, was quickly forgotten.

This small slice of family drama reveals that, after two weeks, sheltering in place is beginning to take its toll. Thus, when the Amazon Fresh order she had placed very early Tuesday morning arrived Thursday evening, Nell thoroughly scrubbed the black-marble-topped “island” in our kitchen before placing any grocery bags on it. She washed all the berries in a colander then put them into a large Tupperware container. She also wiped down every package of food prior to our putting them into their respective storage places. Later that night, meanwhile, as I set up the kitchen for its nightly cleaning so I could watch with Nell the second episode of season one of Broadchurch—which Nell has been asking me to watch for years, if only because of how many actors and actresses who have appeared in Doctor Who are in it—my frustration level boiled over into a series of angry “Oh, for f—k sake!” expulsions. For the record, I am loving the series—its leisurely-unfolding murder investigation and emphasis upon revealing the darker secrets of a supposedly idyllic small town compare favorably to the first season of Twin Peaks.

It does helps tremendously that the weather has been relatively warm and sunny the last few days, and we have three porches opening off our two stories; climbing multiple flights of internal stairs on a regular basis is a good aerobic workout—really, it is. Throwing a stick in our smallish back yard for our soon-to-be-six-year-old golden retriever over and over and over again works as well.

We also have a breathtakingly spectacular view of downtown Boston. Three weeks ago, if we looked through our kitchen window, we would routinely see three or more moving dots of white light as planes took off from Boston Logan International Airport. Now, it is unusual to see even a single plane in the air. That said, I cannot decide if there are fewer lights visible at night in downtown Boston’s office buildings or not.

I think there are fewer lights at night these days.

**********

When I came downstairs on Friday afternoon, there was no schedule on the flip chart—it was a quiet morning in Nell’s classroom—but our younger daughter had livened up that room in our unique way.

White board March 27

Earlier that day, Nell had ventured to our preferred CVS to pick up some prescriptions. This was the first time she had driven her car in 15 days, though I had moved it onto the street a few times so I could use my car—we have tandem parking—and the outing significantly improved her mood. As often as she and the girls go for runs in our neighborhood, sometimes you need actually to go somewhere.

Meanwhile, I was wicked excited to start class at 3 pm because I had prepared what I hoped would be a genuinely fun exercise—one that did not involve coin flipping, die rolling or card shuffling: a 30-question, multiple-choice quiz game.

Quiz Game 1

My heart sang when our younger daughter came out of the disordered cavern she calls a bedroom, took one look at my computer screen—I had again lugged my desktop computer into the classroom—pumped her right arm and exclaimed, “Yes!” Her sister reacted positively as well.

The rules were simple. I alternated which daughter would answer the question— older daughter went first based upon the scientifically-rigorous method Eenie Meenie Minie Moe. The questions covered everything we had discussed in the previous two weeks—political theory, American politics, statistics and the history film noir. Each question was worth one point and had four possible answers, though one answer was deliberately patently absurd; they had the desired effect of making the quiz feel less like work and more like a game. Finally, if one daughter did not answer a question correctly, her sister had the opportunity to answer it.

In the end, after a boisterous 45 minutes of laughter, our older daughter won 16-12, with two points going to Daddy because neither daughter answered two questions correctly. Her “prize,” besides bragging rights, was a giant box of Cheerios I had recently discovered in the revolving cupboard in the kitchen. Huffily reminding me, “I no longer eat cereal, I eat OATMEAL,” she declined her prize, which now sits discreetly on the kitchen counter next to my coffee maker.

There is just no pleasing some people.

And with that our second week of home schooling came to an end.

**********

As I said, we are still figuring out how best to home school our smart and curious daughters. After two weeks of political science and math—not coincidentally, my initial choices for my Yale major—I have settled upon the following tentative weekly schedule:

Monday: Using a single story to illustrate some aspect of American political history/economy

Tuesday: Using the book I am writing to learn about our daughters’ and my Jewish-American heritage

Wednesday: Discussing the history of jazz and rock using my personal collection of DVDs and online tools like Polyphonic  videos.

Thursday: Learning more applied math by examining a wide range of interesting datasets

Friday: Film history and, most likely, additional quizzes.

Onward and, you know, forward we go.

Until next time…please stay safe and healthy…

Organizing by themes I: American politics

This site benefits/suffers/both from consisting of posts about a wide range of topics, all linked under the amorphous heading “data-driven storytelling.”

In an attempt to impose some coherent structure, I am organizing related posts both chronologically and thematically.

Given that I have multiple degrees in political science, with an emphasis on American politics, it is not surprising that I have written a few dozen posts in that field…and that is where I begin.

I Voted sticker

**********

I started by writing about the 2016 elections, many based on my own state-partisanship metric (which I validate here).

The absurdity of the Democratic “blue wall” in the Electoral College

Hillary Clinton’s performance in five key states (IA, MI, OH, PA, WI)

Why Democrats should look to the south (east and west)

How having (or not) a college degree impacted voting

An alternative argument about gerrymandering

An early foray into what I call “Clinton derangement”

The only statistic from 2016 that really matters

**********

Here are a few posts about presidential polling (before FiveThirtyEight jumped on the bandwagon)…

Be careful interpreting President Trump’s approval polls

…and the 2017 special election in Georgia’s 6th Congressional District (GA-6)

Ossoff and the future of the Democratic Party

Using GA-6 polls to discuss statistical significance testing (spoiler: I am not a fan)

**********

And then I started looking ahead to 2018…first to control of the United States House of Representatives (“House”). Note that posts are often cross-generic…

An alternative argument about gerrymandering

The impact of voting to repeal (and not replace) Obamacare (May 2017)

I debut my simple forecast model (June 2017)

Making more points about polls and probability

A March 2018 update

A followup March 2018 update (after which I stopped writing about the 2018 House elections)

…then the United States Senate

The view from May 2017

What it meant that the Senate voted NOT to repeal Obamacare in July 2017

The view from December 2017

…and, finally, races for governor in 2017 AND 2018.

The view from June 2017

A tangentially-related post may be found here.

**********

After Labor Day 2018, I developed models (based on “fundamentals” and polls) to “forecast” the Senate elections…

September 4

September 13

October 23

…and those for governor (the October 23 post addressed both sets of races)

September 16

These culminated in…

My Election Day cheat sheet

And my own assessment of how I did (spoiler: not half bad)

Speaking of assessments, I took a long look at my partisan lean measure here.

And I carefully examined some polling aggregation assumptions here.

**********

Beginning in April 2019, I turned my attention to the 2020 elections.

First came a wicked early look at the relative standings of the dozens of women and men actually or potentially seeking the 2020 Democratic presidential nomination:

April 2019

Then came a wicked early look at the 2020 presidential election itself.

April 2019

And, of course, a wicked early look at races for Senate (2020) and governor (2019-20).

With a post-Labor-Day update.

With the first of regular updates to both the 2020 Democratic presidential nomination and the 2020 presidential election in May 2019

This post both set up the first Democratic debates and had good news for Democrats looking ahead to 2020.

This post set up the second Democratic debates and drew some conclusions about who “won” and “lost” the first debates.

This post updated the data for August 2019 and drew some conclusions about who “won” and “lost” the second debates.

Ditto for September 2019, October 2019, November 2019,  December 2019, January 2020

Once voting commenced in the 2020 Democratic presidentil nomination process, I wrote posts specific to the

As for the 2020 general election, here is the view one year in advance. And two assessment of Emerson College polls (one, two).

**********

Finally, there are other politics posts that defy easy categorization.

I indulged in some speculative alternative history about the presidential elections of 1948 and 2000.

I delineated issue differences between Democrats and Republicans.

I got a bit personal here and here, concluding with the fact that, despite overlapping in the same residential college at Yale for two years, I did NOT know Associate Justice Brett Kavanagh at all.

I argued for the abolition of the Electoral College.

Until next time…

2018 Election Cheat Sheet: How did I do?

I should apologize to our younger daughter’s friend’s mother.

In my…determination…to be settled in front of the television with snacks and beverages at precisely 6 pm EST on November 6, 2018, I might have been a bit abrupt collecting our youngest daughter from a local taqueria where said friend’s mother had generously taken them to supper (after schlepping them and one other girl back from gymnastics class).

However, thanks to help from the same daughter, I was at my post at the appointed time. Our youngest daughter even carefully picked out all of the red M&M’s (plain and peanut) from their decorative bowls. There were no red cashews to extract (but they were still delicious).

I also had a blue mechanical pencil to mark my 2018 Election Guide, as well as an entire 12-pack of unflavored Polar Seltzer cans sitting on the floor to my left (as the evening turned into midnight and beyond, the line of empty blue cans on the floor emanating from the carton grew longer and longer).

And sitting within reaching distance of my right arm was this colorful fowl.

IMG_4010

You know it is a celebration in our home when “the rooster” makes an appearance. Rather than ice water, however, this evening it was filled with blue lagoons—which my wife Nell still cannot decide more closely resembles Windex or Scope.

As the early returns from Indiana and Kentucky were being tabulated on MSNBC, however, a sinking feeling set in that I would not be drinking as much of this cocktail as I had anticipated. I remembered from 2008 that Indiana’s Democratic pockets report much later than its eastern-half Republican counties, but Democratic Senator Joe Donnelly was trailing by well over 20 percentage points in a race that both FiveThirtyEight.com and I had labeled “Lean Democratic.” (Republican Mike Braun would eventually defeat Donnelly by 5.9 percentage points [points]) And Democrat Amy McGrath was not faring as well in the early tallies from the 7th Congressional District (CD) in Kentucky against incumbent Republican Andy Barr as I had hoped. (McGrath would eventually lose by 3.2 points.)

When polls closed at 7 pm EST in Vermont and Virginia, MSNBC almost immediately projected wins in their respective United States Senate (Senate) races for Independent Bernie Sanders and Democratic Senator Tim Kaine—meaning that the first calls of the night were for men I had voted for in 2016 in completely different contexts—Sanders in the Massachusetts Democratic Presidential Primary and Kaine as the Democratic nominee for vice president.

That sinking feeling only grew worse as the FiveThirtyEight.com “live tracker” of Democrats’ chances of regaining control of the United States House of Representatives (House) dipped below 50% around 8:30 or so. Nell, worried, yelled into the living room, “I am not hearing any whoops or cheers.”

At just before 9 pm (when it was already clear Republicans would not only maintain control of the Senate but add seats), the indefatigable Steve Kornacki  announced NBC was giving the Democrats only a 65% chance of regaining the House, projecting they would finish with between 216 (2 too few) and 232 House seats; this translates to a net gain of between 21 and 37 seats.

Finally, however, as votes were counted in Virginia and, especially, New York, both the FiveThirtyEight.com tracker and the NBC “big board” manned brilliantly by Kornacki creeped higher and higher.  I do not remember when MSNBC projected Democrat Abigail Spanberger had defeated two-term Republican Dave Brat in Virginia’s 7th CD, but it was then I realized the anticipated “blue wave” (at least in the House) would materialize. When Democrat Max Rose beat two-term incumbent Republican Dan Donovan in New York’s 11th CD (on Republican-leaning Staten Island), it was off to the races.

Finally, at just before 11 pm EST, MSNBC (OK, I cannot find when they made their call, but it was likely within a few minutes of CNN) projected a Democratic takeover of the House.

A few minutes later, a not-yet-asleep Nell came downstairs to say that one of our politically-like-minded downstairs neighbors had texted her appreciation of my (partially-restrained) whooping-dancing “We got the House! We got the House!”

For the first time since the election of Republican Donald J. Trump as president of the United States, accompanied by a Republican House and Senate, plus a conservative-leaning Supreme Court, I truly exhaled.

**********

In my previous post, I laid out a series of “projected” final margins for 17 (of 35) Senate races and all 36 governor’s races. In this post, I described two simple models of the number of House seats Democrats would net in 2018 based upon the change from 2016 in the Democratic (vs. Republican) margin in the total vote cast nationwide for the House. In 2016, Democrats lost the total national House vote by 1.1 points (while netting 6 seats as they improved by 4.7 points from 2014).

Votes are still being tabulated across the country, especially in California, but enough time has passed since Election Day to see how my projections compared to the actual margins (and to the FiveThirtyEight.com assessment of those same races), starting with the House.

House. According to the indispensable Cook Political Report vote tracker, as of 6 pm EST on November 18, 2018, nearly 110.7 million votes had been cast in House races. For perspective, 81.0 million, 86.8 million and 78.8 million House votes were cast in the last three midterm elections (2006, 2010, 2014), respectively. And that total was 129.8 million in the last presidential election year (2016). (House election data from the Cook tracker and here).

Democrats have thus far won 53.0% of those votes, compared to 45.7% for Republicans (and 1.3% for a smattering of third-party candidates) for a Democratic margin of 7.7 points…and an 8.8-point shift towards the Democrats from 2016 (and 13.5 points from 2014!)

According to my preferred “simple” model (change in margin only), a shift of 8.8 points would yield a gain of 26 seats (and give Democrats a 72% chance of regaining House control). My “complex” model (accounting also for whether the election was a midterm or not) was more bullish on the net seat gain (30) but more bearish on the probability (64%). Averaging across the two models yields a net of 28 seats and a 68% probability of Democratic House control.

Meanwhile, FiveThirtyEight.com’s final House forecasts projected a Democratic national House margin of 9.2 points (the median of their Lite, Classic and Deluxe forecasts) and a net gain of 38 (ditto) seats. Using the FiveThirtyEight.com projected House margin ups my average projected House seat gains to 33 with an 82% chance of regaining control.

With three-seven House races yet to be called, the likeliest outcome is that Democrats will net 38 (36-41) House seats, widely geographically dispersed: six (with Republican David Valadao the likely winner in CD 21) in California; four each in New Jersey and Pennsylvania (+5 D, +1 R); three each in New York and Virginia; two each in Florida, Illinois, Iowa, Michigan and Texas; and one each in Arizona, Colorado, Georgia (with incumbent Republican Rob Woodall leading Democrat Carolyn Bourdeaux by just 419 votes[1]), Kansas, Maine, New Mexico (almost certainly), South Carolina and Washington. Incumbent Republican Mia Love also leads Ben McAdams by just 419 votes. Minnesota showed no net change as Democrats flipped the 2nd and 3rd CDs while Republicans flipped the 1st and  8th CDs.

Based on the information I had on the morning of Election Day, that is 5 (3-8) seats more than I projected Democrats to net, well below the average nine seats by which my models “missed” across 24 previous midterm elections—and consistent with my models underestimating gains/losses in “wave” elections.

FiveThirtyEight.com almost perfectly nailed the actual Democratic net gain of seats, though (as of this writing) they overestimated the Democratic national House margin by 1.5 points; historically, this is not an especially large difference.

Most fascinating, however, is that a net gain of 38 House seats would actually be one seat higher than the upper range of what NBC was projecting at 9 pm EST on Election Day. Vote counting may be laborious and require infinite patience, but it is ultimately rewarding.

Senate. Table 1 compares the actual margin (Democratic percentage of total vote minus Republican percentage of total vote) in 33 2018 U.S. Senate races; italicized states indicate Republican pickups while boldfaced states indicate Democratic pickups. I excluded California, where incumbent Democrat Dianne Feinstein beat fellow Democrat Kevin de Leon by 9.0 points, and the special election in Mississippi, where incumbent Republican Cindy Hyde-Smith will face Democrat Mike Espy in a November 27 runoff. The latter race should be an easy win for Hyde-Smith in ruby red Mississippi (18.5 points more Republican than the nation as a whole, according to my 3W-RDM), but Hyde-Smith’s recent comments may make this race closer than expected.

Table 1. Comparing projected to actual 2018 U.S. Senate election margins*

State 3W-RDM Actual Difference

(Projected – Actual)

AV Difference

(Projected – Actual)

JBWM 538.com JBWM 538.com
Hawaii 34.3 42.2 -11.2 11.2
Vermont 27.7 39.9 -1.3 1.3
Maryland 22.6 33.9 -3.3 3.3
Massachusetts 22.1 24.8 -1.3 1.3
New York 21.6 33.0 4.8 4.8
Rhode Island 18.0 23.0 -5.6 5.6
Connecticut 12.8 20.2 -1.2 1.2
Delaware 12.5 22.2 -4.7 4.7
Washington 12.1 17.0 -5.4 5.4
New Jersey 12.0 10.6 1.5 -0.9 1.5 0.9
New Mexico 6.5 23.5 5.3 5.3
Maine 5.9 19.0 -0.9 0.9
Michigan 2.2 6.6 -6.3 -4.6 6.3 4.6
Nevada 2.0 5.0 4.7 4.0 4.7 4.0
Virginia 1.5 16.0 0.2 0.2
Minnesota SE 1.5 10.6 1.4 1.0 1.4 1.0
Minnesota 1.5 24.1 2.7 2.7
Wisconsin 0.7 10.8 -0.8 -2.0 0.8 2.0
Pennsylvania -0.4 12.8 -2.0 1.3 2.0 1.3
Florida -3.4 -0.2 -2.2 -3.4 2.2 3.4
Ohio -5.8 6.4 -5.7 -5.0 5.7 5.0
Arizona -9.7 2.2 0.7 0.5 0.7 0.5
Texas -15.3 -2.6 3.2 1.9 3.2 1.9
Missouri -15.9 -6.0 -5.5 -7.0 5.5 7.0
Indiana -16.3 -5.9 -7.1 -9.6 7.1 9.6
Mississippi -18.5 -20.3 0.8 0.8
Montana -18.6 3.5 -0.2 -1.2 0.2 1.2
Tennessee -25.8 -10.8 -6.3 -5.4 6.3 5.4
Nebraska -25.8 -19.6 -4.7 4.7
North Dakota -29.4 -10.8 -2.4 -6.0 2.4 6.0
Utah -33.1 -32.2 -2.8 2.8
West Virginia -35.5 3.3 0.0 -4.2 0.0 4.2
Wyoming -45.7 -37.0 7.1 7.1
Average Difference

(all projected elections)

 

-1.7

 

-1.9

 

3.1

 

3.7

Average Difference

(both projections only)

 

-1.7

 

-2.5

 

3.1

 

3.6

      *Excluding California (two Democrats) and the special election in Mississippi (runoff

      November 27, 2018)

States are sorted from most-to-least Democratic, according to their 3W-RDM score. The table presents the numeric and absolute value of the difference between the actual and projected Democratic margins in each election for both JustBearWithMe (JBWM) and FiveThirtyEight.com. Two sets of averages are presented at the bottom of the table: one was calculated using every election projected (I only projected the 17 most “interesting” races, while FiveThirtyEight.com projected all 35) and one was calculated only using the 16 listed Senate elections projected by both JBWM and FiveThirtyEight.com.

With Democratic Senator Bill Nelson conceding to Republican Rick Scott in the Florida Senate race, and the runoff in Mississippi still likely to result in a Republican hold, Democrats appear to have lost a net of 2 Senate seats. Besides Florida, Republicans ousted Democratic incumbents in Indiana, Missouri and North Dakota; they also won hard-fought races in Tennessee and Texas. Democrats, however, beat incumbent Republican Dean Heller in Nevada and won the open seat in Arizona vacated by Republican Jeff Flake.

My final back-of-the-envelope estimate was a loss of 0.9 Senate seats, while the median final FiveThirtyEight.com projection was a loss of 0.5 Senate seats; this is at most a 1.5 seat underestimate, depending on what happens in Mississippi, though I was slightly closer to the actual outcome. Both projections “called” the Florida and Indiana Senate races wrong—while FiveThirtyEight.com called the Missouri Senate race wrong as well.

Both JBWM and FiveThirtyEight.com overestimated Democratic margins in a swath of states stretching from North Dakota (average 4.2 points) south and east to Florida (2.8); states in which both projections overestimated the Democratic margin by at least four points were Ohio (5.4, on average), Michigan (5.5), Tennessee (5.9), Missouri (6.3) and Indiana (8.4). FiveThirtyEight.com also underestimated Republican margins in solidly Democratic Delaware, Hawaii, Rhode Island and Washington, as well as in solidly Republican Nebraska.

At the same time, both projections underestimated Democratic margins in Nevada (4.4) and, to a lesser extent, Texas (2.7); FiveThirtyEight.com also significantly underestimated Democratic margins in New Mexico, New York and Wyoming.

Overall, I overestimated Democratic Senate race margins by an average of 1.7 points (3.1 points in absolute terms) while FiveThirtyEight.com missed by an average of 1.9 points (3.7 in absolute terms). Only looking at the 16 Senate races we jointly assessed, FiveThirtyEight.com’s performance is slightly worse: overestimating Democratic margins by 2.5 points (though just 3.6 in absolute terms). This suggests FiveThirtyEight.com performed slightly better in Senate races in which the winner was clear well in advance.

Governor. Table 2 compares the current actual margin (Democratic percentage of total vote minus Republican percentage of total vote) in 35 2018 gubernatorial elections; italicized states indicate Republican pickups while boldfaced states indicate Democratic pickups. I excluded Nebraska because no polls were conducted of its gubernatorial election. States are again sorted from most-to-least Democratic.

Table 2. Comparing projected to actual 2018 U.S. Gubernatorial election margins**

State 3W-RDM Actual Difference

(Projected – Actual)

AV Difference

(Projected – Actual)

JBWM 538.com JBWM 538.com
Hawaii 34.3 29.0 -4.1 -1.1 4.1 1.1
Vermont 27.7 -15.0 -10.0 -3.6 10.0 3.6
California 23.2 22.6 5.7 5.2 5.7 5.2
Maryland 22.6 -12.7 -8.7 4.9 8.7 4.9
Massachusetts 22.1 -32.6 -2.7 1.4 2.7 1.4
New York 21.6 22.2 0.5 3.1 0.5 3.1
Rhode Island 18.0 15.5 0.1 -4.9 0.1 4.9
Illinois 14.7 15.4 -2.4 6.1 2.4 6.1
Connecticut 12.8 3.2 -3.9 -1.9 3.9 1.9
Oregon 8.7 6.4 -3.0 -0.1 3.0 0.1
New Mexico 6.5 14.4 5.2 5.0 5.2 5.0
Maine 5.9 7.6 -1.8 -4.7 1.8 4.7
Colorado 2.2 10.6 1.9 -1.8 1.9 1.8
Michigan 2.2 9.5 0.5 -0.2 0.5 0.2
Nevada 2.0 4.1 2.9 3.9 2.9 3.9
Minnesota 1.5 11.5 2.7 1.4 2.7 1.4
Wisconsin 0.7 1.2 -2.9 -0.5 2.9 0.5
New Hampshire 0.1 -7.0 -0.8 1.3 0.8 1.3
Pennsylvania -0.4 16.8 0.2 1.4 0.2 1.4
Florida -3.4 -0.4 -4.3 -4.6 4.3 4.6
Iowa -4.7 -2.7 -4.3 -3.5 4.3 3.5
Ohio -5.8 -4.2 -5.4 -5.7 5.4 5.7
Georgia -9.6 -1.4 -0.4 0.8 0.4 0.8
Arizona -9.7 -14.2 -2.5 -0.5 2.5 0.5
Texas -15.3 -13.3 2.6 3.6 2.6 3.6
South Carolina -15.7 -8.0 4.7 5.6 4.7 5.6
Alaska -19.2 -7.9 -5.1 -3.9 5.1 3.9
Kansas -23.4 4.5 7.1 5.8 7.1 5.8
Tennessee -25.8 -21.1 -5.7 -7.5 5.7 7.5
South Dakota -25.8 -3.4 -2.5 -0.9 2.5 0.9
Arkansas -28.2 -33.5 -3.5 -6.1 3.5 6.1
Alabama -28.4 -19.2 1.6 -3.0 1.6 3.0
Idaho -34.2 -21.6 -3.4 -5.2 3.4 5.2
Oklahoma -38.1 -12.1 -2.3 -4.9 2.3 4.9
Wyoming -45.7 -39.8 -4.2 -9.8 4.2 9.8
Average Projected-Actual -1.4 -0.7 3.4 3.5

      **Excluding Nebraska because no polls were conducted of its gubernatorial election

With Democrats Andrew Gillum in Florida and Stacey Abrams (sort of) in Georgia conceding to Republicans Ron DeSantis and Brian Kemp, respectively, Democrats netted six governor’s mansions. Democrats defeated Republican incumbents in Illinois and Wisconsin and won Republican-held open seats in Kansas, Maine, Michigan, Nevada and New Mexico; Republican Mike Dunleavey beat Democrat Mark Begich to win the open Independent-held governor’s mansion in Alaska. At the same time, Republicans cut their losses by narrowly holding the governor’s mansions in Florida, Georgia, Iowa and Ohio.

My final back-of-the-envelope estimate was a Democratic net gain of 9.2 governor’s mansions, while the median final FiveThirtyEight.com projection was 8.2 governor’s mansions. Both projections incorrectly “called” the gubernatorial elections in Florida, Iowa and Ohio for the Democratic candidate while mistakenly projecting a win in Kansas by Republican Kris Kobach over Democrat Laura Kelly.

Both JBWM and FiveThirtyEight.com overestimated Democratic margins by at least three points in Iowa (3.9 points on average), Idaho (4.3), Alaska (4.5), Florida (4.5), Arkansas (4.8), Ohio (5.5), Tennessee (6.6), Vermont (6.8) and Wyoming (7.0)—and, to a lesser extent Connecticut (2.9); all but Vermont[2] are at least 3.4 points more Republican than the nation as a whole. However, both projections underestimated Democratic margins in Nevada (3.4), New Mexico (5.1), South Carolina (5.2), California (5.5) and Kansas (6.5)—and to a lesser extent Texas (3.1); I addressed the woes besetting Kansas Republicans here.

Overall, I overestimated Democratic gubernatorial election margins by an average of 1.4 points (3.4 points in absolute terms) while FiveThirtyEight.com did so by an average of just 0.7 points (3.5 in absolute terms). Clearly, while both forecasts were identical in terms of correct and incorrect “calls,” FiveThirtyEight.com did a better job of assessing election probabilities and final margins.

Summary. Across all 51 Senate and gubernatorial elections “projected” by both JBWM and FiveThirtyEight.com, my projections overestimated Democratic margins by 1.5 percentage points on average, only slightly worse than the FiveThirtyEight.com average overestimation of 1.3 points. This is almost exactly the latter’s overestimation of the total national House Democratic margin by, at most, 1.5 points, suggesting that the 2018 midterm electorate was slightly more Republican than pollsters estimated (though well within historic parameters). The average miss in either direction of 3.4-3.5 points was also well within the range of recent elections.

However, these averages mask wide variation in Democratic under- and over-performance. In races with both a Senate and a gubernatorial election, Democrats had the most disappointing showings in Florida, Ohio and, especially, Tennessee; they also underperformed in Senate races in mostly Democratic states and in gubernatorial elections in mostly Republican states. Underperformance in two traditional presidential swing states—Florida and Ohio—could be of some concern to Democrats as they try to unseat President Trump in 2020.

On the brighter side, states where Democrats overperformed—California, Nevada, New Mexico and Texas—are all in the southwest (as is Arizona, where Democrats won a Senate race for the first time since 1988), an area of the country trending sharply Democratic. The closer-than-expected race for governor in South Carolina plus very close losses for governor in Florida and Georgia may also herald improved Democratic prospects in the southeast.

Besides geography, did state partisanship determine which state electorates were more or less Democratic than anticipated? For FiveThirtyEight.com’s gubernatorial election projections, the answer is…maybe. The Pearson correlation[3] between a state’s 3W-RDM and its numeric difference in gubernatorial margin is +0.44, while for the absolute value of the difference it is -0.37, suggesting that the more Democratic the state, the more Democrats overperformed in that state’s race for governor, while missing less in absolute terms. However, this could simply be an artifact of FiveThirtyEight.com’s newly-minted methodology for projecting gubernatorial elections.

The bottom line. As of January 3, 2019, Democrats will control the U.S. House of Representatives—most likely by 31 seats—for the first time in eight years, despite slightly “underperforming” in the total national House vote (which they still won by nearly 8 points). Their net gain of ~38 seats is the highest Democratic total since the Watergate elections of 1974 (49). Moreover, turnout in House elections—nearly 111 million votes and counting—will be at least 35.2% higher than the average turnout in 2006, 2010 and 2014. Democrats did not regain the Senate—suffering disappointing losses in Florida, Indiana and Missouri (as well as Tennessee and Texas)—but by winning elections in two southwestern states (Arizona, Nevada), they held their losses to two (or one, if they pull off an upset in Mississippi in 18 days), ground they will almost certainly make up in 2020, when the map is more favorable to Democrats (or, at least, far less unfavorable). Finally, they netted six governor’s mansions (including holding on to win a closer-than-expected race in Connecticut), despite disappointing losses in Florida, Georgia, Iowa and Ohio. Democrats will control governor’s mansions in 23 states—the most since the 2008 elections—which have a combined 280 electoral votes, meaning more than half of the nation’s population will have a Democratic governor.

Do not let a few disappointing results fool you. The Democratic wave in 2018 was strong and wide.

Until next time…

[1] We actually know Ms. Bourdeaux’s sister from our younger daughter’s former ballet class; following our move, we also share a dog park.

[2] Vermont voters may not have wanted to tell pollsters—in just three public polls—they were unwilling to vote for transgendered Democratic nominee Christine Hallquist.

[3] A number from -1.0 to +1.0 indicating the strength of the linear relationship between two variables. Briefly, a positive correlation means that as one variable increases the other variable does the same (and vice versa), while a negative correlation means that as one variable increases the other variable decreases (and vice versa). A correlation of zero means there is no association at all.

A plea to readers with two weeks until Election Day 2018 ends…

The 2018 midterm elections end in two weeks, on November 6, 2018.

I write “end” because early voting is underway in 28 states, including Massachusetts. In fact, it opened Monday, October 22, and so I dragged our two daughters to Brookline Town Hall so they could participate in the process. And, yes, I voted straight Democratic with the exception of governor.

The best habits start early as our youngest daughter’s backpack reveals.

I Voted sticker.JPG

Along those lines—as a former political-scientist-in-training, lifelong political junkie and huge fan of democracy, I cannot strongly encourage you enough to vote.

Please.

This plea applies both to my American readers and to my many international readers, whenever the opportunity next presents itself.

**********

I do three things in this post.

  1. Update analyses of 2018 elections for the United States House of Representatives (“House”), United States Senate (“Senate”) and governor.
  2. Attempt to quantify the Republican polling “bounce” following the September 27, 2018 Senate Judiciary Committee testimony by Dr. Christine Blasey Ford and United States Court of Appeals Circuit Judge Brett M. Kavanaugh.
  3. Reconsider House, Senate and gubernatorial election projections under two scenarios: one where polls underestimate Republican voting by 3 percentage points, and where polls underestimate Democratic voting by 3 percentage points.

**********

Updated analyses. As of Tuesday afternoon, October 23, 2018, the FiveThirtyEight forecast was that Democrats would win the national House vote by 8.9 percentage points. According to my “simple” model, that translates to an 89.8% probability Democrats net at least the 23 House seats they need to regain control of the House (projecting a 29 seat gain). By comparison, the FiveThirtyEight forecast is 85.8% and 40 seats—reasonably close to my less “complex” estimates.

Since I last wrote about Senate races, I created two new metrics.

  1. A weighted probability of Democratic victory
  2. A projected Democratic election day margin.

The victory probability is simply a weighted average of the “fundamentals” and adjusted polling average (APA) probabilities, with the latter increasing in weight based upon the number, recency and quality of published polls. I estimate the “fundamentals” probability by assuming a normal distribution whose standard deviation is that of my 3W-RDM measure (4.9), and I estimate the APA probability using a margin of error derived from the total sample size of all polls of each election conducted entirely in calendar year 2018, to which I add 3.0 to account for recent average polling bias (averaging across the last four elections in the table “Polling bias shifts from election to election”).

Weights are calculated using this formula:

#Polls/10 + #Sept/Oct Polls/2 + (Average Pollster Rating – 4.3) + %Sept/Oct Polls/10

For example, 51 total polls have been conducted since January 1, 2018 in the Florida Senate race, with 21 conducted since September 1, with an average pollster rating of 2.7 (using the letter-grade assigned by FiveThirtyEight on a scale where A+=4.3, A=4.0, etc.). Thus, the amount by which polls are weighted over fundamentals in this race is 51/10 + 21/2 + (2.7-4.3) + 41.2/10 = 5.1 + 10.5 – 1.6 + 4.1=18.1.

The “projected Democratic margin” is also the weighted average of the “fundamentals” and APA margins.

Table 1: Democratic Victory Probabilities and Margins in 10 Key 2018 Senate Elections

State Probability Democratic Victory Projected Democratic Margin Democratic Gain, Hold, Loss 3W-RDM
AZ 90.3% D+2.5 Gain R+9.7
FL 70.2% D+1.3 Hold R+3.4
IN 72.8% D+1.2 Hold R+16.3
MO 38.9% R+0.4 Loss R+15.9
MT 92.4% D+4.1 Hold R+18.6
NV 43.6% R+0.2 Hold D+2.0
ND 0.2% R+7.3 Loss R+29.4
TN 19.2% R+2.7 Hold R+25.8
TX 0.1% R+6.2 Hold R+15.3
WV 90.4% D+5.1 Hold R+35.5
  Lose 0.8 seats R+0.3 R+1 R+16.8

The rough-and-ready forecasts in Table 1 are consistent with anything from a Democratic loss of one seat to a Democratic gain of one seat, depending on outcomes of very close races in Missouri and Nevada (not to mention Florida, Indiana and, perhaps, Tennessee). In this, they are broadly in agreement with the FiveThirtyEight Senate forecast (19.0% chance Democrats regain Senate; average loss 0.5 seats), though they are far more bullish on Democratic chances in Missouri (61.1%), North Dakota (30.1%), Tennessee (24.5%) and Texas (21.5%), and more bearish on Arizona (63.4%).

Not to belabor the point, but given the extreme “redness” of these 10 states (16.8 percentage points more Republican than the nation, on average), even a net loss of “only” one Senate seat would be a moral victory of sorts for Democrats…though a net gain of two or more seats would be an actual victory, in that they would then control the Senate.

Table 2: Democratic Victory Probabilities and Margins in 19 Key 2018 Gubernatorial Elections

State Probability Democratic Victory Projected Democratic Margin Democratic Gain, Hold, Loss 3W-RDM
AK 18.8% R+2.3 Loss R+19.2
AZ 1.5% R+8.6 Hold R+9.7
CO 99.8% D+9.0 Hold D+2.2
CT 100.0% D+9.0 Hold D+12.8
FL 99.4% D+4.6 Gain R+3.4
GA 38.2% R+0.2 Hold R+9.6
IL 100.0% D+16.5 Gain D+14.7
IA 95.5% D+2.7 Gain R+4.7
KS 31.4% R+2.0 Hold R+23.4
ME 100.0% D+6.8 Gain D+5.9
MI 99.9% D+9.7 Gain D+2.2
MN 99.8% D+8.7 Hold D+1.5
NV 53.3% D+0.9 Gain D+2.0
NM 100.0% D+8.6 Gain D+6.5
OH 28.2% R+0.3 Hold R+5.8
OK 0.5% R+6.8 Hold R+38.1
OR 100.0% D+7.9 Hold D+8.7
SD 23.6% R+4.4 Hold R+25.8
WI 99.1% D+5.0 Gain D+0.7
AVE Gain 7.9 seats D+3.4 D+7 R+4.3

Table 2 presents Democratic victory probabilities and margins for those gubernatorial elections most likely to change partisan hands and/or with margin< 10 percentage points. This group of states is far more purple, averaging only 4.3 points more Republican than the nation as a whole.

The governor’s race in Alaska altered considerably on October 19, when Independent Governor Bill Walker suspended his reelection campaign and endorsed Democrat Mark Begich over Republican Mike Dunleavy, though the likely outcome (a Dunleavy win) remains the same. Otherwise, Democrats remain strongly favored to pick up governor’s mansions in Florida, Illinois, Iowa, Maine, Michigan, New Mexico and Wisconsin, losing only in Alaska (Walker was effectively a Democrat). Extremely close races in Georgia, Nevada and Ohio could go either way, while Democrats are within shouting distance in Kansas and South Dakota (albeit, with only two polls). At the same time, once-possible pickups in Arizona and Oklahoma now seem far less likely.

The bottom line (again, in broad agreement with FiveThirtyEight) is that Democrats appear poised to net between six and nine governor’s mansions, putting them tantalizingly close to a majority.

A Kavanaugh bounce? There is evidence of a pro-Republican bounce in polling following the sequence of events between the Judiciary hearings on September 27 and the final confirmation vote (50-48 in favor) on October 6, including the week-long FBI investigation, spurred by increased Republican enthusiasm and voting likelihood.

To quantify the bounce, I compared Senate and gubernatorial race polls, unskewed and weighted by pollster rating, conducted before (though after August 1) and after September 27; all polls had to be completed by September 26 or started no earlier than September 27.

Table 3: 2018 Polling Data in 16 Key 2018 Senate Elections, Before and After Ford-Kavanaugh Hearings 

State Adjusted Poll Average

8/1-9/26

Adjusted Poll Average

9/27-10/22

Difference

(Pre-Post)

3W-RDM
AZ D+3.0 (11) D+0.3 (8) -2.8 R+9.7
FL D+0.3 (14) D+2.1 (10) +1.9 R+3.4
IN D+1.9 (3) D+0.2 (8) -1.6 R+16.3
MI D+14.7 (9) D+12.4 (3) -2.3 D+2.2
MN D+5.9 (4) D+10.1 (3) +4.2 D+1.5
MS R+13.5 (1) D+1.4 (1) -14.9 R+18.5
MO R+1.6 (7) R+0.8 (7) -0.8 R+15.9
MT D+5.2 (6) D+3.7 (1) -1.5 R+18.6
NV D+0.2 (5) R+1.6 (5) -1.8 D+2.0
NJ D+7.2 (2) D+6.9 (5) -0.3 D+12.0
ND R+4.6 (1) R+12.9 (2) -8.3 R+29.4
OH D+12.2 (6) D+16.5 (2) +4.3 R+5.8
PA D+15.3 (7) D+14.4 (1) -0.9 R+0.4
TN D+0.3 (8) R+6.2 (5) -6.5 R+25.8
TX R+3.2 (10) R+7.0 (7) -3.8 R+15.3
WV D+8.2 (7) D+7.9 (4) -0.3 R+35.5
WI D+7.9 (4) D+9.7 (2) +1.8 D+0.7
AVE D+4.4 D+2.4 -2.0 R+10.4

On average across 17 key Senate races (Table 3), the Republican position in the polls improved by an average of 2.0 percentage points following the Ford-Kavanaugh hearings. And the more Republican the state, the more the Republican candidate’s position improved (r=0.48)—as can be seen in Arizona, Mississippi, North Dakota, Tennessee and Texas (and also, surprisingly, in Democratic-leaning Michigan and Nevada). In fact, removing six states where the Democrat is strongly favored (albeit, four won by Republican presidential nominee Donald Trump in 2016; average 3W-RDM D+1.7), the Republican increase jumps to 3.7 percentage points (D+1.2 to R+2.6; r=0.31). At the same time, the bounce fades (-0.6; r=0.41) once you examine only states with at least two polls in both time periods.

Table 4: Polling Data in Selected 2018 Gubernatorial Elections, Before and After Ford-Kavanaugh Hearings

State Adjusted Poll Average

8/1-9/26

Adjusted Poll Average

9/27-10/19

Difference

(Pre-Post)

3W-RDM
AK R+0.9 (2) R+11.8 (2) -10.9 R+19.2
AZ R+6.5 (9) R+14.5 (6) -8.0 R+9.7
AR R+36.7 (1) R+37.7 (1) +1.0 R+28.2
CA D+10.6 (7) D+11.4 (4) +0.8 D+23.2
CO D+9.1 (2) D+7.5 (1) -1.6 D+2.2
CT D+8.5 (4) D+5.7 (3) -2.8 D+12.8
FL D+4.6 (13) D+4.7 (7) +0.1 R+3.4
GA D+1.9 (3) R+1.8 (6) -3.7 R+9.6
IL D+15.1 (4) D+17.6 (2) +2.5 R+16.3
KS R+0.4 (3) R+0.1 (1) +0.3 R+23.4
ME R+0.6 (1) D+7.8 (2) +8.4 D+5.9
MD R+16.9 (3) R+18.7 (2) -1.8 D+22.6
MA R+36.5 (2) R+38.8 (1) -2.3 D+22.1
MN D+6.9 (4) D+10.5 (3) +3.6 D+1.5
MI D+11.0 (8) D+11.2 (2) +0.1 D+2.2
NV D+2.1 (2) R+0.9 (4) -3.0 D+2.0
NH R+12.7 (2) R+13.7 (3) -1.0 D+0.1
NY D+1.0 (1) D+22.7 (2) +21.7 D+21.6
OH R+2.0 (5) D+1.3 (2) +3.3 R+5.8
OR D+7.0 (2) D+5.1 (1) -1.9 D+8.7
PA D+15.8 (6) D+11.4 (1) -4.4 R+0.4
RI D+7.3 (1) D+9.9 (2) +2.6 D+18.0
SC R+7.8 (2) R+23.7 (1) -15.9 R+15.7
TN R+14.5 (7) R+18.4 (3) -3.9 R+25.8
TX R+17.0 (8) R+20.1 (4) -3.1 R+15.3
WI D+3.3 (6) D+4.6 (2) +1.3 D+0.7
AVE R+1.8 R+2.6 -0.8 R+1.1

Alabama, Idaho, Iowa, New Mexico, Oklahoma have no polls after September 26

Hawaii had no polls between August 1 and September 26.

The trend was similar in 26 governor’s races (Table 4; average R+1.1)—an overall Republican increase of 0.8 percentage points, though once you remove New York (only one extreme outlier poll between August 1 and September 26), the increase becomes 1.7 percentage points. Again, the sharpest increases were in more Republican states (r=0.43), especially Alaska, Arizona, Georgia, South Carolina, Tennessee and Texas (and, surprisingly, in purple-to-blue Connecticut, Nevada and Pennsylvania). Examining only states with at least two polls in both time periods, the Republican increase jumps to 1.5 percentage points (r=0.36).

So, the “Kavanaugh bounce” appears to have been roughly one-to-three percentage points, and it was most evident among Republican voters in Republican states—who may well have been “coming home” to their party anyway (the Ford-Kavanaugh hearings may only have started the process earlier). And there is evidence the bounce is fading somewhat—at least in House voting (which covers the entire nation rather than a Republican-leaning set of states). The FiveThirtyEight House forecast dropped from an 80.7% chance of a Democratic takeover on September 30 to 73.9% on October 4—but then started to increase again October 9. Similarly, the forecast was a 32.0% chance of a Democratic Senate takeover on September 30, but by October 11 the probability had dropped to 18.6%. After rising three percentage points since then, as of Tuesday afternoon, October 23, it stood at 18.9%; the gubernatorial forecast does not lend itself to an analogous comparison.

Alternate polling scenarios. That even a small Kavanaugh “bounce” was enough to reduce Democratic Senate and gubernatorial gains by one-to-two seats shows how close this election (or, at least, the binary outcome of “majority/minority status”) is.

This can be shown by increasing—or decreasing–every polling margin by three percentage points, consistent with the statistical “bias” polls have displayed in the last four even-numbered election years; the direction of that bias changes from year to year.

For the House, if the projected national Democratic margin in total vote was actually 5.9% (that is, a 7.0% election-to-election increase), the probability they regain control plummets to 25%, with an average net gain of only 20 seats, three fewer than necessary. By contrast, however, were the margin 11.9%, Democrats would be locks to regain House control (99.6% probability), netting an average of 40 seats. Put simply, this close to Election Day, Democrats could still fall achingly short of a House majority—or net as many as 20 more seats than necessary.

For the Senate, a pro-Democratic polling bias of three percentage points in the polls would result in losing seats in Florida, Indiana, Missouri and North Dakota, while gaining zero seats; this is the nightmare scenario for Democrats. And while a pro-Republican polling bias of “only” two percentage points would mean winning in Arizona, that would still be a net loss of three Senate seats.

By the same token, a pro-Republican polling bias of three percentage points would almost certainly give them majority status in the Senate, as they still lose Heidi Heitkamp’s seat in North Dakota while winning seats in Arizona, Nevada and (possibly after a recount) Tennessee.

That is, this close to Election Day, a range of losing four Senate seats and gaining two seats remains plausible for Democrats.

Finally, in governor’s races, Democrats appear to be far enough ahead in key states that even a pro-Republican polling bias of three percentage points would still net them five governor’s mansions (win in Florida, Illinois, Maine, Michigan, New Mexico, Wisconsin; lose in Alaska) with Iowa a virtual tie. But a pro-Democratic polling bias of three percentage points would truly unleash a blue gubernatorial tsunami: not only would they likely WIN in Alaska (and Iowa), they would most likely add Georgia, Kansas, Nevada, Ohio, Oklahoma and South Dakota to their column. An historic net gain of 13 governor’s mansions could easily be in the offing.

**********

One overarching message from this barrage of data is that while pollsters do their best to model an unknown electorate and reduce uncertainty—the actual set of citizens who will turn out to vote remains, at best, a highly-educated guess and uncertainty (beyond just margin of error) still remains. Still, some good news for Democrats lies buried in a recent New York Times/Siena College poll. While the overall result was an eight percentage point lead for Republican Senator Ted Cruz (and among those whose certainty to vote is confirmed by prior voting behavior), Democratic challenger Beto O’Rourke actually LED by three percentage points among those who said they were almost certain to vote.

The other overarching message, then, is simply that every vote counts—even the tiniest changes in the composition of the 2018 electorate could fundamentally who governs us for the next two years.

I cannot say this often or loudly enough…PLEASE VOTE!

Until next time…

An update on projected 2018 Democratic U. S. House seat gains

UPDATED Midnight EST, November 20, 2018. As of this writing, Democrats have netted 38 seats in the United States House of Representatives, with three races still to be called. Democrat Ben McAdams narrowly leads incumbent Republican Mia Love in Utah’s 4th Congressional District (CD), while Democrats trail narrowly in California’s 21st and Georgia’s 7th CD. For a fuller analysis of the 2018 midterm elections, see here. For more details on called and uncalled races, see here and here.

**********

With Democrat Conor Lamb’s narrow victory over Republican Rick Saccone in the March 13, 2018 special election to fill the United States House of Representatives (House) seat vacated by Republican Tim Murphy, there were 238 Republican-held seats, 193 Democratic-held seats and four open seats (2 Democratic, 2 Republican).

The two Democratic open seats (MI-13, NY-25) are likely to remain open until the entire House is up for election on November 6, 2018. In the last three presidential elections, the Democratic nominee won these Congressional districts (CD) by an average 67.4 and 18.3 percentage points (points), respectively, so it is highly likely both seats will be won by Democrats. The seat vacated by Republican Trent Franks (AZ-08) will be filled in a special election on April 24, while that of Republican Patrick Tiberi (OH-12) will be filled in a special election on August 7. The last three Republican presidential nominees won these two CDs by 22.7 and 10.2 points, respectively. The Arizona seat will likely remain Republican, though the Ohio seat could be close.

For now, however, let us assume that party control of these four CDs does not change. That would leave the partisan balance at 195 Democrats and 240 Republicans. This means that Democrats need to flip a net total of at least 23 seats to win a House majority after the 2018 midterm elections.

In a previous post, I listed projected Democratic net House seat gains based upon four models; I developed two of these models.

Briefly, I use the change in the margin by which Democrats won/lost the total nationwide vote for all 435 House seats (Democratic % – Republican %) from the previous Congressional elections (24 election, 1970-2016). In 2016, the Democrats lost the national House vote 47.6% to 48.7%, for a margin of -1.1 points[1]. The current FiveThirtyEight estimate is that Democrats leading in generic Congressional polls (“If the election was held today, would you vote for the Democratic or Republican candidate in your district?”) by 6.5 points (13.1% unsure/other parties).  If that were the actual election margin, that would equal a change of 6.5 – (-1.1) = 7.6 points.

In my “simple” ordinary least squares (OLS) regression model, I use only this value:

Estimated Democratic net seat gain = -1.63 + 3.11 * Change in Democratic margin

On average, every one percentage increase in the change in Democratic margin nets Democrats 3.1 additional House seats. Plugging a 7.6 point change in Democratic margin into the formula yields:

-1.63 + 3.11 * 7.6 = 22.01

This model has Democrats falling one seat shy of a House majority, even after winning the national House vote by 6.5 points—a result that should give believers in “one person, one vote” pause.

The 95% confidence interval (CI) around this estimate is 13.3 – 30.8, meaning we can say with 95% confidence that a 7.6 point shift in Democratic national House margin would result in a net gain of between 13 and 31 seats. This translates to a 41.2% chance the Democrats net at least 23 House seats in November 2018.

Again—that is when Democrats win the national House vote by 6.5 points. Take your pick of what to blame: incumbency advantage (Republicans gained a net 63 House seats in 2010 before the most recent round of redistricting), geographic self-sorting (Democrats in urban areas, Republicans in rural/small town areas, suburbs up for grabs) and partisan gerrymandering.

In a slightly more complex model, I account for whether the Congressional election coincided with a midterm or a presidential election:

If Midterm: Estimated Democratic net seat gain = -2.77 + 3.41 * Change in Democratic margin + 2.433

If Presidential: Estimated Democratic net seat gain = -2.77 + 2.08 * Change in Democratic margin

And here I make a confession.

In previous discussions of results from this complex model, I used an incorrect “intercept” term (-1.63 instead of -2.77), increasing estimated net Democratic House seat gains (by 1.14) and probabilities of House recapture.

Not a huge change, but I do strive for accuracy here.

Cutting to the chase, under the assumed 7.6 point change in national Democratic House margin, the complex model yields an estimated Democratic net gain of 25.6 seats (95% CI: -8.2 to 59.3[2]) with a 56.0% probability of regaining control of the House.

Here is Table 1 from my previous post updated to reflect the corrected “Berger 2” model and a required 23 net House seat gain for Democrats.

Table 1: Projections of 2018 Democratic net gains in House seats

Dem national House margin Abramson Brennan Berger 1 Berger 2
2% 19 5 8

(0%)

10

(12%)

4% 23 7 14

(1%)

17

(33%)

6% 27 13 20

(28%)

24

(52%)

8% 30 15 27

(78%)

31

(66%)

10% 34 21 33

(97%)

37

(75%)

11% 36 28 36

(99%)

41

(78%)

12% 38 31 39

(99+%)

44

(81%)

14% 42 41 45

(99+%)

51

(85%)

16% 46 56 52

 (100%)

58

 (88%)

And here is my updated graphic displaying the probability Democrats recapture the House given various changes in Democratic margin  (and dotted line indicating House control is 50-50):

Figure 1: Probability Democrats control U. S. House of Representatives after 2018 elections given change in Democratic margin from 2016

Democratic Probability 2018 House capture

You may ask why I assessed change in margin rather than actual margin. The two values are correlated r=0.55, meaning that there is a relatively strong, linear association between them (i.e., more often than not, when one increases/decreases the other increases/decreases), so the models should be broadly similar.

The outcome of interest to me is whether Democrats net at least 23 House seats they did not win in 2016. For that to happen, Democratic 2016 margins must increase from, say, 0 to -15 points to between +15 to 0 points. Republicans won 31 House seats in 2016 by ≤15 points, so a uniform swing (i.e. margin in every CD changes an identical amount) of 15 points toward the Democrats would net them eight more House seats than they need.

But let’s say the swing is not uniform. Perhaps the 195 Democratic-held seats see no change in margin in 2018, while the 240 Republican-held seats see a uniform shift of 15 points. That equates to an 8.3 point overall margin shift toward the Democrats, still netting them 31 seats (by comparison, my two models show an 8.6 point change in Democratic margin netting them 24-28 House seats).

Now, I could simply compare the actual Democratic margin in the national House vote to the actual number of seats won. The simple model is:

Estimated Democratic seats = 217.92 + 4.61 * Democratic margin

And the more complex model is:

If Midterm: Estimated Democratic seats = 211.29 + 4.18 * Democratic margin + 9.877

If Presidential: Estimated Democratic seats = 211.29 + 5.90 * Democratic margin

Plugging the current FiveThirtyEight estimate (Democrats+6.5) into either model yields an estimated 248 Democratic House seats—a net gain of 53 seats! That is a far more optimistic projection that those estimated using change in Democratic margin (22-26).

Moreover, if the Democrats and Republicans simply break even in 2018 (1.1 point increase from 2016), the “actual margin” models would project a bare Democratic House majority (218-221 seats), a net gain of 23-26 seats. By contrast, the “change in margin” models would project a net Democratic gain of only 2-3 seats under this assumption.

Statistically speaking, all four models are valid and account for more than 80% of the variance in the independent variable (net seat gain, seats won), a very high value for such simple models. Various indicators,[3] meanwhile, suggest that the simpler models are a better fit to the data than the complex models.

For now, though, I stick with the “change in margin” models as they align more with other projections (and a back-of-the-envelope seat-by-seat examination) and have, to me, a stronger conceptual underpinning.

That leaves one final test of the “change in margin” models.

One way to test the reliability (measuring the same underlying concept over repeated measurements) of OLS regression models is to remove, say, 1970 data, then rerun the regression. You would use the new model to estimate the value for the missing data point. Doing this for all (or a random subset) of data points yields illustrative comparisons between actual and estimated value, as shown in Table 2.

Table 2: Estimated (after removing that year’s data) and actual net Democratic change in House seats, 1970-2016

Year % Point Change in Dem  Margin Actual Net Dem House Seats Estimated –Actual Net Dem House Seats Model 1 Estimated –Actual Net Dem House Seats Model 2
1970 6.8 12 8.2 12.6
1972 -3.4 -13 0.8 3.6
1974 11.4 49 -17.9 -13.6
1976 -3.3 1 -13.6 -12.3
1978 -4.7 -15 -1.3 -1.5
1980 -6.0 -35 15.8 25.5
1982 9.1 27 -0.4 4.5
1984 -6.6 -16 -6.7 -0.6
1986 4.8 5 8.9 12.4
1988 -2.1 2 -10.6 -10.2
1990 0.1 7 -8.7 -7.6
1992 -2.8 -9 -1.4 0.5
1994 -12.0 -54 17.9 16.6
1996 7.1 3 19.1 12.4
1998 -1.2 4 -9.8 -9.2
2000 0.5 1 -1.1 -3.0
2002 -4.3 -8 -7.4 -7.8
2004 2.0 -2 6.9 3.8
2006 10.5 31 0.1 5.6
2008 2.6 23 -17.4 -22.8
2010 -17.2 -63 11.0 6.6
2012 7.9 7 17.6 9.9
2014 -7.1 -12 -12.8 -14.5
2016 4.7 6 7.5 1.2
Average, Raw Values 0.2 0.5
Average, Absolute Values 9.3 9.1
Average, Absolute Values, Midterms Only 8.7 9.4
Average, Absolute Values, Presidential Only 9.9 8.8

In the final two columns, a positive value means Democrats underperformed their estimated net House seat gain, while a negative values means they overperformed. Democrats underperformed and overperformed on both models 11 times each. In 1982 and 1992, the Democrats slightly overperformed on the simple model and slightly underperformed on the complex model.

The largest underperformances (≥10.0 in either model) were in 1980, 1994, 1996, 2010 and 2012; three were wave years (1980, 1994, 2010) in which Democrats averaged a net loss of 54 House seats. Similarly, the largest overperformances were in 1974, 1976, 1988, 2008 and 2014; strong waves occurred in 1974 and 2008 (Dems+36, on average) and 2014 (Dems-12). The pattern is not perfect, however, as the model was fairly close in the wave year of 2006 (Dems+31), missing by an average of three seats. Still, in 10 of 24 elections, at least one model missed the actual net change in Democratic House seats by ≥10 seats.

On average, estimates were spot on, as overperformance and underperformance essentially cancel out. However, when you examine the absolute value (difference in either direction) of differences, the models do worse, averaging +/- 9 seats. The slight differences between midterm and presidential election years make little practical difference.

Applying these average differences to the current projections (change from 2016 in Democratic national House margin of 7.6 points) yields

Berger 1: 22.0 +/- 9.3 = 12.7 to 31.3 net Democratic House seats

Berger 2: 25.6 +/- 9.1 = 16.5 to 34.7 net Democratic House seats

Put another way: Democrats would need to win the national House vote by 9.8 points in the simple model, and by 8.5 points in the complex model, for the lower end of these projected ranges to be 23 seats (what Democrats need to net to control the House after the 2018 midterm elections).

This is not how a representative democracy is supposed to work.

Until next time…

[1] Using data from Dave Liep’s indispensable Atlas of U.S. Presidential Elections

[2] This extremely wide CI results from using only 24 data points to estimate three parameters.

[3] Among them adjusted r-squared, residual sums of squares, degrees of freedom.

Projected 2018 Democratic U.S. House seat gains

This piece (only available to subscribers) appeared earlier today on Taegan Goddard’s absolutely essential Political Wire.

A new Brennan Center report says “extreme gerrymandering” could cost Democrats control of the House unless they ride a massive blue wave.

Because of maps designed to favor Republicans, Democrats would need to win by a nearly unprecedented nationwide margin in 2018 to gain control of the House of Representatives. To attain a bare majority, Democrats would likely have to win the national popular vote by nearly 11 points. Neither Democrats nor Republicans have won by such an overwhelming margin in decades. Even a strong blue wave would crash against a wall of gerrymandered maps.

Yet this is misleading without also mentioning the “the great sorting” of voters that has taken place over the last two decades. An equal, if not bigger, barrier to Democrats winning the House is the extreme urbanization of Democratic voters which leads to millions of “wasted” votes.

 Pew Research study shows:

Voters in urban counties have long aligned more with the Democratic Party than the Republican Party, and this Democratic advantage has grown over time. Today, twice as many urban voters identify as Democrats or lean Democratic (62%) as affiliate with the GOP or lean Republican.

Overall, those who live in suburban counties are about evenly divided in their partisan loyalties (47% Democratic, 45% Republican), little changed over the last two decades.

In addition, while mapping technology has made it easier for congressional maps to be gerrymandered in the redistricting process, the sorting of the electorate into distinct geographic areas makes it even easier.

Both phenomena — the use of gerrymandering during redistricting and the geographic sorting of voters — coexist and give Republicans an advantage in congressional elections.

What does this mean for the 2018 midterm elections? The Cook Political Report has found that in the last three election cycles, Democrats have won roughly 4% fewer seats than votes received nationally. If this trend holds true in 2018, then Democrats would need to win the House popular vote by roughly 7% to win the 23 seats they need to take a majority. This is a similar to a projection made by Emory University political scientist Alan Abramowitz.

In contrast, the Brennan study, which looks at responsiveness of vote margins in individual states, suggests Democrats would need an 11% margin to take control of the House.

I have expressed mild skepticism about the role gerrymandering has played in the maintenance of Republican majorities in the United States House of Representatives (House) since 2011, writing this about “wasted” votes (which I call “extraneous votes” [ExV]):

In 2016, Democrats averaged 112,222 ExV and Republicans averaged 98,582, meaning Democrats averaged 13.8% more ExV than Republicans. Narrowing the analysis only to the 39 states where partisan redistricting is even possible closes the gap: 111,401 to 102,963, with Democrats averaging 8.2% more ExV. Further removing seats with candidate(s) of only one major party reduces the absolute gap to 97,701 to 89,970, with Democrats averaging 8.9% more ExV than Republicans.

This is additional evidence for the geographic self-sorting of Democrats, which I agree (along with the creation of majority-minority legislative districts under the Voting Rights Act) has enabled Republican gerrymandering. I would also observe that Republicans won a net total of 63 House seats (and House control) in 2010, before the current legislative district lines were drawn.

The piece concluded with a tabulation of projected 2018 Democratic net House seat gains given a range of Democrats national House vote margins using the “Abramson” and “Brennan” models. Democrats need a net gain of 23 House seats to recapture the majority.

Given my own research into the relationship between national House vote margin and House seats won (using change in Democratic share of the national House vote from two years earlier; Democrats lost the national House vote by 1.1 percentage points in 2016, despite netting six seats), I decided to append my projections to Goddard’s table. “Berger 1” uses percentage point change only, while “Berger 2” also adjusts for midterm vs. presidential election. For my projections, I display both the estimated net seat gain as well as the probability Democrats net the 23 seats necessary to regain House control. In each column, boldfaced values represent Democratic House control.

Table 1: Projections of 2018 Democratic net gains in House seats

Dem national House margin Abramson Brennan Berger 1 Berger 2
2% 19 5 8

(0%)

11

(14%)

4% 23 7 14

(1%)

18

(36%)

6% 27 13 20

(28%)

25

(55%)

8% 30 15 27

(78%)

32

(68%)

10% 34 21 33

(97%)

39

(77%)

11% 36 28 36

(99%)

42

(80%)

12% 38 31 39

(99+%)

45

(82%)

14% 42 41 45

(99+%)

52

(86%)

16% 46 56 52

 (100%)

59

 (89%)

My projections fall in between those of Abramson and Brennan: Berger 1 requires Democrats to win the national vote by 6.8 percentage points, while Berger 2 requires a margin of only 5.4 percentage points (see Figure 1 below).

Figure 1: Probability Democrats Control U.S. House of Representatives After 2018 Elections Based Upon Change in Democratic National House Vote Share, 2016-18

Democratic Probability 2018 House capture

As of this writing (7:24 pm EST, March 26, 2018), the FiveThirtyEight estimate of Democratic advantage in the generic ballot is 5.7 percentage points (46.0 to 40.3%, down from a high of 13.3 percentage points on December 26, 2017). If that is the actual national House vote margin on November 6, 2018, Democrats would be projected to net between 12 and 27 House seats, depending on the model, meaning they either fell well short of their goal, or just eked it out.

Still, it is a sign of the current lopsided state of American electoral geography that a political party needs to win the national vote by between 4 and 11 percentage points just to break even in House seats.

Until next time…

Positively pondering pesky probabilities, perchance

One inspiration to start this “data-driven storytelling” blog was the pioneering work of Nate Silver and his fellow data journalists at FiveThirtyEight.com; their analyses are an essential “critical thinking” reality check to my own conclusions and perceptions. Indeed, when I finally get around to designing and teaching my course on critical thinking (along with my film noir course), the required reading would include Silver’s The Signal and the Noise and a deep dive into Robert Todd Carroll’s The Skeptic’s Dictionary. I will also include Ken Rothman’s Epidemiology: An Introduction; what drew me to epidemiology (besides my long career as a public health data analyst) was its epistemological aspect. By that I mean how the fundamental methods and principals of epidemiology allow us to critically assess any narrative or story.

To that end, I have been reading with great interest Silver’s 11-part series that “reviews news coverage of the 2016 general election, explores how Donald Trump won and why his chances were underrated by most of the American media.” And while I highly recommend the entire series of articles, the September 21 conclusion is the jumping off point for my own observations about assessing the likelihood of various events.

**********

Let me begin with a passage from that article:

In recent elections, the media has often overestimated the precision of polling, cherry-picked data and portrayed elections as sure things when that conclusion very much wasn’t supported by polls or other empirical evidence.

I personally think investigative journalists are heroic figures who will ultimately save American democracy from its current self-induced peril. But they are trained in a very specific way: deliver the fact of a story with certainty and immediacy. In so doing, they are responding to media consumers with little patience for complex narratives suffused with uncertainty.

To quote Silver again, “a story can be 1. fast, 2. interesting and/or 3. true — two out of the three — but it’s hard for it to be all three at the same time.”

One narrative that developed fairly early about the 2016 presidential election campaign was that Democratic nominee Hillary Clinton was the all-but-inevitable victor. I wrote about one version of this flawed narrative here.

Reinforcing this narrative were election forecasts issued during the last weeks of the campaign that practically said “stick a fork in Trump, he is finished.” But as Silver rightly observes, some of these models were flawed because they failed to account for the “correlation in outcomes between [demographically similar] states.” For example, were Republican nominee Donald Trump to outperform his polls in Wisconsin on Election Day, he would likely also do so in Michigan, Minnesota and Iowa. And that is essentially what happened.

Still, because aggregating polls yields a more precise picture of the state of an election at a given point in time, I aggregated these 2016 election forecasts. Going into Election Day, here were some estimated probabilities of a Clinton victory, ranked lowest to highest.

FiveThirtyEight 71.4%
Betting markets 82.9%[1]
The New York Times Upshot 84.0%
DailyKos 92.0%
HuffingtonPost Pollster 98.2%
Princeton Election Consortium (Sam Wang) 99.5%

The average and median forecast was 88.0%. Remove the most skeptical forecast (though Clinton still a 5:2 favorite), and the average and median jump to 91.3% and 92.0%, respectively. By contrast, if you remove the least forecast, the average and median drop to 84.1% and 83.5%, respectively.

It is an understandable human tendency to look at a probability over 80% and “round up” from “very likely, but not guaranteed” to “event will happen.” And, under the frequentist definition of probability, we would be correct more than 80% of the time in the long run.

But we would not be correct as much as 20% of the time.

Ignoring Wang’s insanely optimistic forecast for various reasons, the “aggregate” forecast I had in mind on Election Day was that Clinton had about an 84% chance of winning.

The flip side, of course, was that Trump had about a 16% chance of winning.

A good way to interpret this probability is to think about rolling a fair, six-sided die.

Pick a number from one to six. The chance that if you roll the die, the number you picked will come up, is 1 in 6, or 16.7%.

On Election Day, Trump metaphorically needed to roll his chosen number…and he did.

But even if take the Wang-inclusive average of 88%, that is still a 1 in 8 chance. Throw eight slips of paper with the numbers one through eight written on them in a hat (I like fedoras, myself), pick one and draw. If your number comes up (which will happen 12% of the time over many draws), you win.

Trump picked a number between one and eight then pulled it out of our hypothetical fedora, and he won the election.

One way people misunderstand probability (and one of many reasons I am resolutely opposed to classical statistical significance testing) is mentally converting event x has a very low probability (like, say, matching DNA in a murder trial—only a 1 in 2 million chance!) with that event cannot happen.

So, even the Wang forecast—which gave Trump only a 1 in 200 chance of winning—did NOT mean that Clinton would definitely win. It only meant that Trump had to pull a specific number between one and 200 out of our hypothetical fedora. He did, and he won.

**********

On the other end of the spectrum is an overabundance of caution in assessing the likelihood of an event. This usually occurs when interpreting election polls.

In this post, I discussed Democratic prospects in the 2017 and 2018 races for governor.

One of the two governor’s races in November 2017 is in Virginia, where Democratic governor Terry McAuliffe is term-limited. The Democratic nominee is Lieutenant Governor Ralph Northam, and the Republican nominee is former Republican National Committee chair Ed Gillespie.

Here are the 13 public polls of this race listed on RealClearPolitics.com[2] taken after the June 13, 2017 primary elections:

Poll Date Sample MoE Northam (D) Gillespie (R) Spread
Monmouth* 9/21 – 9/25 499 LV 4.4 49 44 Northam +5
Roanoke College* 9/16 – 9/23 596 LV 4 47 43 Northam +4
Christopher Newport Univ.* 9/12 – 9/22 776 LV 3.7 47 41 Northam +6
FOX News* 9/16 – 9/17 507 RV 4 42 38 Northam +4
Quinnipiac* 9/14 – 9/18 850 LV 4.2 51 41 Northam +10
Suffolk* 9/13 – 9/17 500 LV 4.4 42 42 Tie
Mason-Dixon* 9/10 – 9/15 625 LV 4 44 43 Northam +1
Univ. of Mary Washington* 9/5 – 9/12 562 LV 5.2 44 39 Northam +5
Roanoke College* 8/12 – 8/19 599 LV 4 43 36 Northam +7
Quinnipiac* 8/3 – 8/8 1082 RV 3.8 44 38 Northam +6
VCU* 7/17 – 7/25 538 LV 5 42 37 Northam +5
Monmouth* 7/20 – 7/23 502 LV 4.3 44 44 Tie
Quinnipiac 6/15 – 6/20 1145 RV 3.8 47 39 Northam +8

Eight of these polls have Northam up between four and seven percentage points, including four of the last six. Two polls show a tied race. No poll gives Gillespie the lead.

And yet, here was the headline on Taegan Goddard’s otherwise-reliable Political Wire on September 19, 2017, referring to the just-released University of Mary Washington (Northam +5) and Suffolk polls (Even): Race For Virginia Governor May Be Close.

Granted, the two polls gave Northam an average lead of only 2.5 percentage points, which, without context, suggest a close race on Election Day. Furthermore, all three Political Wire Virginia governor’s race poll headlines since then have been on the order of: Northam Maintains Lead In Virginia.

Here is the thing, however. Most people (as I did) will equate “close” with “toss-up.” But there is a huge difference between “we have no idea who is going to win because the polls average out to a point or two either way” and “one candidate consistently has the lead, but the margin is relatively narrow.”

The latter is clearly the case in the 2017 Virginia governor’s race, with Northam’s lead averaging 4.4 percentage points in eight September polls within a narrow range (standard deviation [SD]=3.3). We are still more than five weeks from 2017 Election Day (November 7), so this is unlikely to be “herding,” the tendency of some pollsters to adjust their demographic weights and turnout estimates to avoid an “outlier” result (undermining the rationale for aggregating polls in the first place).

The problem comes when members of the media try to interpret the results of individual polls. They have absorbed the lesson of the “margin of error” (MoE) almost too well.

For example, the Monmouth poll conducted September 21-25, 2017 gives Northam a five percentage point lead, with a 4.4 percentage point MoE. Applying that MoE to both candidates’ vote estimates, we have 95% confidence that the “actual” result (if we had accurately surveyed every likely voter, not a sample of 499) is somewhere between Gillespie 48.4, Northam 44.6 (Northam down 3.8) and Northam 53.4, Gillespie 39.6 (Northam up 13.8). It is this range of possible outcomes, from a somewhat narrow Gillespie victory to a comfortable Northam win that leads members of the media to imply through oversimplification that this race will be close, meaning “toss-up.”

And yet, even within this poll, the probability (using a normal distribution, mean= 5.0, SD=4.4) that Northam is as little as 0.0001 percentage points ahead is 87.2%, making him a 7:1 favorite, about what Hillary Clinton was on Election Day 2016.

OK, maybe that was not the best example…

But when you aggregate the eight September polls, the MoE drops to about 1.3[3], putting the probability Northam is ahead at well over 99%. Even if the MoE only dropped to 3.0, the probability of a Northam lead would still be about 93%.

My point is this. Every poll needs to be considered not just as an item in itself (polls as NEWS!) but within the larger context of other polls of the same race. And in the 2017 Virginia governor’s race, the available polling paints a picture of a narrow but durable lead for Northam.

I have no idea who will be the next governor of Virginia. But a careful reading of the data suggests that, as of September 29, 2017, Lt. Governor Ralph Northam is a heavy favorite to be the next governor of Virginia, despite being ahead “only” 4 or 5 percentage points.

**********

Finally, here is an update on this post about the Democrats’ chances of regaining control of the United States House of Representatives (House) in 2018.

Out of curiosity, I built two simple linear regression models. One estimates the number of House seats Democrats will gain in 2018 only as a function of the change from 2016 in the Democratic share of the total vote cast in House elections. The Democrats lost the total 2016 House vote by 1.1 percentage points, so if they were to win the 2018 House vote by 7.0 percentage points, that would be an 8.1 percentage point shift.

Right now, FiveThirtyEight estimates Democrats have an 8.0 percentage point advantage on the “generic ballot” question (whether a respondent would vote for the Democratic or the Republican House candidate in their district if the election were held today).

My simple model estimates a pro-Democratic House vote shift of 9.1 percentage points would result in a net pickup of 26.7 House seats, a few more than the 24 they need to regain control. The 95% confidence interval (CI) is a gain of 17.0 to 36.4 seats.

But the probability that Democrats net AT LEAST 24 House seats is 71.1%, making the Democrats 5:2 favorites to regain control of the House in 2018.

My more complex model adds a variable that is simply 1 for a midterm election and 0 otherwise, as well as the product of this “dummy” variable and the change in Democratic House vote share. I hypothesized (correctly) that this relationship would be stronger in midterm elections.

This model estimates that a 9.1 percentage point increase from 2016 in the Democratic share of the House vote would result in a net gain of 31.8 seats. However, with two additional independent variables (and only 24 data points), the 95% CI is much wider, from a loss of 7.0 seats to a history-making gain of 68.3 seats.

Still, this translates to a 66.1% probability (2:1 favorites) the Democrats regain the House in 2018.

Figure 1 shows the estimated probability the Democrats regain the House in 2018 using both models and a range of percentage point changes in House vote share from 2016.

Figure 1: Probability Democrats Control U.S. House of Representatives After 2018 Elections Based Upon the Change in Democratic Share of the House Vote, 2016-18

Democratic Probability 2018 House capture

The simple model (blue curve) gives the Democrats no chance to recapture the House in 2018 until the pro-Democratic change in vote share reaches 6.5 percentage points, after which the probability rises sharply and dramatically to a near-certainty at the 10.0 percentage point change mark. The more complex model (red curve), meanwhile, assigns steadily increasing chances for the Democrats, flipping to “more likely than not” at the 7.0 percentage point change mark; even at a truly historic 15 percentage point change, the complex model only gives the Democrats an 85.3% chance to recapture the House in 2018.

For the record, I lean toward the more complex model.

It is worth noting that in the current FiveThirtyEight estimate, 15.8% of the electorate is undecided or chose a third party candidate (when an option). If the undecided vote breaks heavily toward the party not controlling the White House in a midterm election (one way electoral “waves” form), a 66-71% would likely be an underestimate of the Democrats’ chances of regaining control of the House in 2018.

And…apropos of nothing…Happy 51st Birthday to me (September 30, 2017)!!

Until next time…

[1]  To be honest, I do not recall where I got this number from…possibly from fivethirtyeight.com or maybe from https://betting.betfair.com/politics/us-politics/&#8230;

[2] Accessed September 28, 2017

[3] The total number of voters sampled across these eight polls is 4,915, which is 9.85 times higher than the 499 sampled in the Monmouth poll. The square root of 9.85 is 3.14. Dividing 4.1 by 3.14 gives you 1.31.