When Failure Is Success…And Vice Versa

It was likely in 2001 – though it may have been just after I moved into my new apartment in the Philadelphia suburb of King of Prussia in February 2003 – I received this handsome piece of engraved metal from my more off-than-on-again girlfriend.

When I first read the question, I tried earnestly answer it – until I realized the obvious answer: nothing.

Just as the answer would be “nothing” if the last word was “succeed.”

It is the uncertainty of outcome that makes a thing worth doing. The thrill comes in succeeding when success was not guaranteed.

Moreover, I now think the question is purely hypothetical. I cannot imagine an activity where failure is not an option, no matter how seemingly banal or minor it is. Despite what the self-help gurus and the rah-rah-artists and the well-meaning leaders tell us:

Failure is ALWAYS an option.


In a recent post, I presented an update on the process of publishing the book – Interrogating Memory: Film Noir Spurs a Deep Dive Into My Family History…and My Own – I completed in late January. To summarize, I queried 100 literary agencies between February 5 and May 12, understanding this to the best route to securing a mass-market publisher. To date, 90 agencies have either formally rejected me (22) or have not responded to me within their stated time frame (68). My expectation is the remaining 10 simply have not yet rejected me, though I obviously do not know for certain.

At first glance, though, 100 seems like a large enough number of agencies that success should all but guaranteed. After all, I only need one, right?

Well, the likelihood of at least one agency accepting me as a client depends upon two things: the probability of being accepted by each individual agency, and the extent to which those probabilities are statistically independent; the latter being a fancy way to ask whether being turned down by one agency implies being turned down by other agencies.

Being of a quantitative bent, I calculated this probability as best I could, primarily to reassure myself as the days, weeks and months passed. To simplify matters, I assumed a constant probability of acceptance across all agencies AND the decision Agency A makes has nothing to do with the decision Agency B makes which has nothing to do with the decision Agency C makes…and so forth.

These are not great assumptions: it was obvious I fit better with some agencies than with others, meaning my probability of acceptance likely varied across the agencies. And while the agencies themselves want to you think each makes its own “highly subjective” (to quote nearly every rejection e-mail) decisions regarding new clients, the reality – apparent from a close reading of agent wish lists, emphasizing diverse voices, indifference to “serious” non-fiction, pre-existing platforms, “similar” (high sales) books and the constant refrain of how many queries each receives – is that nearly every agency is approaching those queries through a broadly similar lens.

Still, as a first approximation, we use this formula:

P(Acceptance by 1 or more agencies) = 1 – (1-p)n,

…where p is the probability of being accepted by any given agency, and n is the number of “trials,” in this case 100.

Going into this process, I naively thought my educational background (“I am an EXPERT!”) and the fact of completion (“No worries about me not finishing here!”) would boost my chances. Maybe to, you know, 1 in 100 – or p=0.01.

Well, that translates to a P of 63.4%. Even with 100 queries, the odds were only about 5:3 in my favor.

Lowering my expectations to p=0.005 – 1 in 200 – lowers P to just 39.4%, or about 3:2 against.

Lowering them further to the more realistic levels I should have understood in February (or in July 2017):

At p=0.001, P=9.5%.

At p=0.0005, P=4.9%.

At p=0.0001, P=1.0%.

And so on.

Now, this is when the bluntly American “can-do” mindset responds with “Well, then, you need to keep querying agencies.”

And that is not unreasonable. Except, I had just queried (excepting one with no e-mail address) every agency in WRITER’S MARKET 2019 that met my basic criteria: no reading fees, represents adult non-fiction, accepting new clients.

It can seem noble never to accept failure – to keep trying despite the long odds one faces because there is nothing we cannot do if we blah blah blah – but math does not lie.

Failure is ALWAYS an option.

But, for the sake of argument, let us say I found 100 more agencies to query. Here are the corresponding increases in P:

At p=0.01, P=86.6% (still 13.4% chance of failure).

At p=0.005, P=63.3% (still 36.7% chance of failure).

At p=0.001, P=18.1%.

At p=0.0005, P=9.5%.

At p=0.0001, P=2.0%.

You get the idea. Even at the rose-colored glasses probability of 1% and 200 queries, the probability of at least one acceptance is only slightly better than Hillary Clinton’s chances of beating Donald J. Trump going into Election Day 2016.

Clinton lost that election.

Failure is ALWAYS an option.


But, of course, so is success. There will always be a non-zero probability of both outcomes, no matter how much we – and, again, this is a particularly American perspective – try to “round” to 0 and 1.

Besides, all of this math – as wonderful as math is – misses the larger point: successes sometimes turn out to be failures, and failures sometimes turn out to be successes – at least when considered in the future.

Here are two examples from my own life – I encourage you to do the same with your own lives – where “success” and “failure” proved remarkably fluid.

  1. The unfinished doctorate.

In May 1988, I graduated with a BA in political science from Yale University. That September, I began a one-year stint as a Research Assistant in the Government Department at the Brookings Institution in Washington, DC. I had enjoyed an unpaid internship there two summers earlier, so I was excited return for a salaried position – my first full-top “adult” job.

I hated it from day one; the success of getting the job later became a clear failure when I was let go the following May. Long before then, though, I had applied to five doctoral programs in government – University of California Berkeley, Harvard, University of Michigan, Stanford, Yale – and been accepted, with generous financial incentives, at all five of them.

Wow, I thought, I am set. Like, golden.

I chose Harvard, moving to the Boston suburb of Somerville in late August. For the first two years, I loved being at Harvard – my fellow students were both impressive and friendly, the classes were excellent, and I felt as home as I had my first few days at Yale.

But as oral and written examinations loomed in early June 1991 – ending the master’s degree portion of the program – something imperceptibly shifted. My romantic life was a bit of a mess, for one thing. Also, I had miscalculated at the end of 1990, when it was the turn of the students who had enrolled at the same time as me to provide the entertainment for the holiday party. We decided to use Saturday Night Live as the frame for our skits – complete with guest host monologue.

In a truly “what the hell was I thinking?” moment, I decided I would deliver the monologue as my academic advisor, Professor Gary King, who had just achieved full tenure at the age of 30. It was a fairly gentle bit of mockery – revolving him stopping wearing ties once he received tenure – but, in retrospect, it may have been unwise to satirize the soon-to-be-chair of my doctoral committee. Next thing I know, I barely pass my examinations – to this day, I think Gary blindsided me during the oral exams when he questioned me about my chosen area of interest, electoral geography.

Still, I put together a doctoral committee – including friend and mentor, Yale Professor David Mayhew – and wrote a dissertation proposal. It was accepted, and I set to work collecting data – even driving to Concord, NH in May 1992 to photocopy town-level results from that state’s 1976 presidential primaries. I wrote some early chapters.

But the joy was vanishing. I had difficulty translating my theory of “differential trait salience”[1] into mathematical models – and articulating it to my committee and fellow students. Moreover, I insisted on applying this model not to general elections – with their highly stable and complete data – but to presidential primary elections – with their highly unstable and incomplete data. I rationalized my creeping sense of failure by quipping bitterly, “Gary’s idea of advising is: go off, do some stuff, bring it back to me, and I’ll tell you why it’s wrong.”

This was garbage, by the way. King was an excellent – albeit socially awkward – political scientist and teacher. I was just not ready to listen, lacking maturity, humility and discipline. It is also likely my yet-to-be-diagnosed depression was kicking in – or kicking in harder – and I began to spend a lot of time in this excellent restaurant only two blocks from my apartment:

As I write in Chapter 11 (A Film Noir Fan is Born), “Credit card receipts reveal I spent at least $418.94 in 1991, $856.40 in 1992 and $554.79 in the first six months of 1993 there; the sum of $1,830.13 equates to $3,335 in 2019—on a modest academic stipend supplemented by teaching and research assistant work.” Self-medicating, much?

A temporary reprieve from my misery came late in June 1993, when there was a knock on the door that opened from the second-floor apartment I shared with three other 20-somethings onto the interior stairwell of our Somerville triple-decker. Two attractive younger women stood there. I recognized the one on the left as one of the female Harvard seniors who had just moved into the third floor apartment for the summer. The one on the right (one of her roommates) – an adorable brunette of just below medium height wearing glasses and a t-shirt advertising Squeeze’s Babylon and On tour – I recognized from the Greenhouse Café in the Science Center. In fact, almost as soon as I opened the door, I pointed to her, smiled and said, “I know you.”

I do not remember what they needed, but within a few days, the Squeeze fan and I had begun to date. To say she saved my life is overly melodramatic, but our rapidly progressing relationship gave me the strength to make one last push to complete my dissertation. Early the following year, I applied for – and received – a Mellon Dissertation Completion Grant. I even began to “joke” to – well, Nell calls her my first wife, so let’s go with that – “Got the Mellon, can’t elope.”

But it was all for naught. In the spring of 1995, after a disastrous search for a university teaching position,[2] I made the hardest decision of my life: to resign, ABD, from Harvard. My last-ever day as a doctoral student – or so I thought – was June 30, 1995.

For the next decade or so, I thought of this – and, in a way, the seven years prior to it – as the greatest failure of my life. Heck, I did not even have a Master’s Degree to show for it, despite completed the requirements; in the spring of 2015, I finally received that A.M.

Here is the thing, though. I am now thrilled I did not pursue an academic career in political science. Does the end of my six years at Harvard still sting? Absolutely. But do I regret not having to deal with the “publish or perish” nature of academia, with its petty squabbles and bureaucratic nonsense. Heck yes!

Even as I was ending my time at Harvard, first-wife and I found an apartment just a few blocks from the triple-decker we had briefly shared. We moved in over the summer; she had since graduated Harvard and enrolled in a doctoral program at the Massachusetts Institute of Technology, where she focused on atmospheric chemistry, earning her doctorate in four years. Not only is she one of the warmest people I have ever known, she is one of the most brilliant.

That summer, I worked a mundane data analysis job, and I was happier than I had been in years. After a disastrous stint in the Registrar’s Office at Brandeis University, in early October 1996 I landed my first health-related data analysis job at now-defunct Health and Addictions Research Inc. (“HARI”) in downtown Boston. This launched a six-position, 19-year career – ending in June 2015. Even with its abrupt end, I am immensely proud of this career – and the long-term friendships it yielded. But this professional “success” only happened because I “failed” to obtain a doctorate at Harvard.

2. Selling my mother’s condominium

On the evening of August 11, 2004, I stepped from the SEPTA commuter train onto the Radnor station platform. Descending the few steps to the parking lot where I had left my car that morning, I noticed some police officers clustered near my car. Walking closer, I realized they were standing by my car. As I approached, one asked if this was my car. Yes, I replied. That is when I understood someone had broken into my car – literally bending back the front passenger side window of my Buick Century from its rubber frame – and stolen the radio and some other ephemera. They eventually arrested the thief, and I testified against him in court, but did not recover my stolen property.

That weird, roller coaster day – I had had a terrible ice cream date then met a fascinating young woman while waiting for that very same train in Suburban Station – was the low point of one of the lowest periods of my life.

Backing up slightly, after meeting the woman whose gift opens this essay, I ended my relationship with first-wife in late November 2000. Yes, this was cause and effect. In early February 2001, I returned to Philadelphia. Four months later, I began a series of increasingly-important positions in the Research Department of what was then called the Family Planning Council of Southeastern Pennsylvania (“FPC”), the best professional profession I have ever had. The next few years exemplified “lucky in money, unlucky in love.” Gift-woman and I pursued a tempestuous, ill-defined long-distance friendship/romance that confused everyone, even us. This surrounded short-term flings that went nowhere.

Still, things looked promising early in 2003. I moved into the King of Prussia apartment, I was earning a good living, and the Phillies showed promise after some great off-season moves. But just one year later, in early January, my mother’s ovarian cancer returned with a vengeance. On March 1, 2004 – after a few weeks of hospice – Elaine Kohn Berger died at the age of 66. At the tender age of 37, I was an orphan.

Grief does strange things to people. On the day of my mother’s funeral – when I apparently drank most of a bottle of whisky, prompting a friend of my stepfather Eddie nicknamed Yo to declare, “If you try to drive home, I’ll rip out your fucking distributor cap” – my stepfather’s married step-granddaughter (my step-step-niece?) was clearly trying to seduce me. OK, I was not exactly fending off her advances; she was wicked hot. Nothing – much – happened, though she did put a bug in my ear about needing to cut bait and get on with my life. Realizing she was – not wrong – I ended my relationship with gift-woman in the most brutal telephone conversation I have ever had. I lied about my feelings, among other things. Even now, as I write this, I am filled with regret. Not that I ended the relationship, but the utter cruelty with which I did so.

And while all that unfolded, I was trying to settle my mother’s estate. For her own reasons, she had made Eddie and me co-executors. Embittered – and jealous of my relationship with my mother – Eddie decided to contest the will. He hired a lawyer, I hired a lawyer – and a 16-month ordeal began. The sticking point was a condominium my mother owned. She was living there when she and Eddie began dating around 1994 or so. They married in 1997, but my mother continued to earn rent from the condominium. When she died, I began to collect that rent – clearing $1,100 a month. I do not really understand why this made Eddie so upset – maybe grief, maybe the brain tumor that felled him a few years later – but he would not relent.

Flash forward to early December 2004. Yet another short-term relationship had come to a crashing halt, and I was beginning to see the writing on the wall at FPC – they were going in a more qualitative direction, my beloved projects were ending, and there was no room for me to advance. Meanwhile, trips to western Massachusetts the previous two summers had reminded me how much I missed the Boston area.

Even though I had done nothing wrong – other than be an absolute jerk to a woman I loved, for the second time in four years – I felt like an utter failure, trapped and lonely.

Then, soaking in the bathtub one Friday night, I had a brainstorm: why not sell the condominium, split the profits and end the standoff? With my proceeds, I could move to Boston, study biostatistics or epidemiology – maybe finally get that damned doctorate.

I presented the idea to my lawyer, who presented it to Eddie’s lawyer, who presented it to Eddie. Who – to everyone’s astonishment – agreed.

Huzzah! I cried, if only metaphorically.

The first six months of 2005 are a blur now – other than feeling absolute liberation and optimism. The condominium sold fairly quickly. I narrowed my choices to two schools of public health: Harvard, which seemed a bad idea, and Boston University (“BUSPH”), about which I had heard good things at a HARI reunion the previous summer. I arranged to retake my GRE’s. Having missed the deadline to apply to their epidemiology doctoral program, I applied to the one in biostatistics. Deciding I had been away from “higher math” too long, I was instead accepted into their master’s program. Which was fine; the process would just take a few years longer.

In March, a chance meeting at my local laundromat turned into a much-needed, if necessarily short-term, romance. I literally told her “I am moving to Boston in September” within minutes of meeting her. Looking back, she was the perfect transition relationship – even if she did move to Boston a few months after I did. That got – weird, though only briefly.

On June 30, 1995 I tearfully ended my game-changing four years at FPC. In August, I drove to Boston to find a new apartment, settling on a complex in Waltham not that different from the one in King of Prussia. Having not yet received my share of the settlement, though, I was forced to borrow the necessary first-last-security deposit payments from a close Yale friend. He graciously obliged.

Finally, at the end of August, I drove to a lawyer’s office in Philadelphia, where I was given a check for – let’s just say it was low six-digits. I immediately paid off – well, my Yale friend – my student loan debts and three credit cards, keeping only the Discover Card. It pays you back, you know.

The rest is wonderful, serendipitous history. Four days after moving to Waltham (with laundromat-woman) – hiring a moving company for the first time – the used Buick Century Eddie had given me when I moved to Philadelphia died. On September 6, I wrote a check for something like $34,000 to Cambridge Honda so I could drive away in a brand new black 2005 Honda Accord. Still in great shape, I hope to pass it on to our older daughter in a few years. I settled happily into my new classes, though I had to drop one – four was just too many; I finished the MA in three semesters, not two, as I had planned.

And on Halloween night 2005, a radiant elementary school teacher named Nell wrote to me on Friendster – and that is how I met my wife. She pretty much had me when she used “Persiflage” as the subject of her first e-mail to me.

In short, had I not reached a point of utter despair – grieving the loss of two women I loved, sensing the end of my most rewarding professional job, seeing no end to the fight with my stepfather – I would not have made the drastic, albeit smart in retrospect, decision to sell my mother’s condominium. Had I not made that decision, I would not have returned to Boston, earned both an MA and a PhD, bought my beloved Honda and met my astonishing wife. And without Nell, there are no incredible daughters.

And no Just Bear With Me…or Interrogating Memory.

Failure may always be an option, but it can also have a way of leading to successes, just as seeming successes can end up feeling like failures.

Now, back to the work of getting my book published!

Until next time…please be safe and healthy – and if you not already done so, please get vaccinated against COVID-19!

[1] Essentially, the idea that the overall demographic composition of a geographic area – a state, a county, a Congressional district – determined which demographic traits were most politically salient within an individual. This acknowledged that each of us has a race AND an ethnicity AND a socioeconomic status AND a religion (or no religion) AND an age AND a marital status AND an education level AND so forth. More often than not, race is the primary predictor of partisanship. But if an area is, say, 95+% Non-Hispanic White then a trait like education level or religion might be the primary predictor. Or something like that – I have not thought deeply about in more than 25 years.

[2] The disastrous – at least for a tried-and-true Democrat like me – 1994 midterm elections hurt as well. I realized how difficult it was going to be to separate my strong partisan lean from my need for professional objectivity.

Measuring the Unmeasurable: Ranking One’s Favorite Music, Part 1

I recently updated a data-based discussion of my cinematic “guilty pleasures,” adding a comparison of “most-acclaimed” and “my favorite” films from a given year or years. In so doing, though, I side-stepped the question of determining with something approaching academic rigor just what my favorite films are, relying solely on my gut to select a favorite film or films from each period.

Readers of this website know that I am fascinated by the art and science of measurement, be it the “noirness” of film festivals, Charlie Chan films, the Marvel Cinematic Universe or baseball player performance. Each of these prior analyses, however, is purely objective: all of the data I used are publicly-available, so the only “subjective” decisions I needed to make were selecting which data to compile and what statistical methods to use. And even when I was analyzing data for which I am the lone source – like this gorgeous distribution of iTunes tracks[1] by year and genre – the only decisions I made related to visualization, not personal preference.

Now, at last, I tackle the deceptively simple question of what music, movies, etc. I like most. More to the point, I address how I can most effectively and efficiently derive a “score” for each track or film, so that I can not only rank order them, but aggregate them into, say, albums, artists and genres, overall or by time period.

In the first installment of this seriees, we journey from my first-ever “mixtape” to my initial attempt to assign scores to my favorite music.


While researching my book Interrogating Memory: Film Noir Spurs a Deep Dive Into My Family History…and My Own, I reread a hand-written “Journal” – really just a paper-clipped set of mid-sized lined pieces of paper ripped from a notebook – I began on May 29, 1981, as I was about to finish 9th grade at Harriton High School.

Buried within the June 5, 1981 entry is this:

Went to Ludington [Library in Bryn Mawr, PA]. … Then from there to Sam Goody’s for The Moody Blues Long Distance Voyager and Phil Collins Face Value. Good stuff. … Then it was Mad’s for Kraftwerk Autobahn.

At the time the two record stores sat a short walk from each other on E. Lancaster Avenue in the Philadelphia suburb of Ardmore. Four decades later, I still own these very albums.

A few months earlier, I had convinced my father – not exactly flush with cash much of the time – to buy three albums for me: The CarsThe Cars and Panorama, and, if memory serves, Supertramp’s Breakfast in America. I still have Panorama on vinyl, though I long since replaced the other two with CDs.

In the 12 months before that, meanwhile, I acquired Fleetwood Mac’s Tusk, Peter Gabriel (III) and Steve Winwood’s Arc of a Diver. I still have Tusk and Arc on vinyl, with PGIII replaced by a CD. Finally, in late July or early August 1981, I acquired Foreigner 4…which I foolishly sold a few years later to buy a new issue of Billboard.

I played these 10 albums – and others I owned – on a turntable, complete with mid-sized brown wooden-cabinet speakers, my mother had bought just over four years earlier. Two moves later, it had migrated to my bedroom, where it sat on a low white wooden shelf with my record collection.

Exactly when I received my first Sony Walkman – and when I got the idea to record a set of tracks I like onto a Maxell cassette – I could not tell you. Nor could I tell you exactly what day in August 1981 – it was likely a Saturday, as I was a day camp junior counselor on weekdays – I placed one of the brown speakers on the floor next to my portable cassette recorder, cued up “Spanish Dancer” on Arc of a Diver, and hit Record. I sat in absolute silence as that track – followed by 13 others – recorded monaurally with zero Dolby noise reduction.

At least, I think it was these 14 tracks, in this order:

Side 1

Spanish Dancer                     Steve Winwood

Night Train                           Steve Winwood

Urgent                                    Foreigner

Juke Box Hero                       Foreigner

Moving in Stereo                  The Cars

All Mixed Up                        The Cars

Side 2

Touch and Go                       The Cars

Running to You                    The Cars

In the Air Tonight                Phil Collins

The Voice                              The Moody Blues

Gemini Dream                      The Moody Blues

Sisters of the Moon              Fleetwood Mac

Games Without Frontiers   Peter Gabriel

I Don’t Remember               Peter Gabriel

I have long since lost the cassette and lined insert card on which I wrote track titles and artists. A thorough search of the cardboard box on the floor to my left – containing dozens of mix cassettes and CDs – did not reveal it. It is purely memory that conjures up this list, though it is a very reasonable list.

My memory also says it was a 60-minute cassette, except this version of Side 1 is 32 minutes long, while Side 2 is 36 minutes long. So, unless I cut off two minutes of a track on Side 1 and have too many tracks on Side 2, this was more likely a 90-minute cassette, and I either ran out of tracks to record (VERY unlikely), or I am forgetting tracks.

Either way, as the cardboard box implies, what I creatively called My Stuff was only my first adventure in mix-making. Four months later I filled a 90-minute cassette with 22 tracks recorded from the radio; I called this cassette Stuff Vol. I. Over the next two years, I completed Stuff Vol. II through XI. With the exception of Side 1 of Stuff Vol. IV, a collection of late 60s/early 70s rock I recorded in June 1982,[2] these were “acquisition” mixes – mostly from the radio but sometimes from borrowed records. Indeed, only six of the 188 (or more, see below) unique tracks I recorded onto those mixes were recorded twice: “I Don’t Remember,” “Tainted Love” by Soft Cell, and the four borrowed-record tracks: two by A Flock of Seagulls, “Kids in America” by Kim Wilde and “Escalator of Life” by Robert Hazard and the Heroes.

It is pretty clear where my musical tastes lay then – and now.


I first wrote here about the 300+ mix cassettes, CDs and videocassettes I recorded between August 1981 and August 2016. These mixes contain at least 3,383 unique tracks.

  • Besides My Stuff, I no longer have Stuff Vol. V (from which I recall seven tracks) and one of the two mixes between Stuff Vol. VIII and Stuff Vol. XI. The other one is in a different plastic case with a sticker reading “I92–Stranglers/Fixx/Devo.” As I have no memory of the “missing” cassette – and I played the heck out of these mixes – it is very likely I simply misnumbered them.
  • When I first began to enter mix contents into a spreadsheet in December 1992, I equated studio and live versions of the same track (except when I didn’t), while including tracks from which I only recorded snippets, or which got cut off at the end.

I had actually been rank-ordering my favorite music for years – summoning from my gut then playing personal “top 25 songs” lists since 1980, if not earlier. In January 1990, meanwhile, I commemorated the end of the 1980s by determining – purely through thought and memory – my 40 favorite tracks of that decade; I mistakenly included “Prime Time” by The Tubes, even though Remote Control, its parent album, was released in 1979. “Promised You a Miracle” by Simple Minds was #1, beating out “All Roads Lead to Rome” by The Stranglers. In March, I purchased my first CD player, allowing me to record from CD to cassette, followed in April by my first PC. The former purchase triggered a wave of CD buying, mostly through buy 1, get 11 for one penny deals; I stocked up on “Best of” CDs. I also began prowling through used record, tape and CD stores. With a PC, meanwhile, I now had a spreadsheet program, though not Microsoft Excel.

By December 1992, I had created 88mixes – excluding the two Top 40 of the 1980s cassettes (10 tracks over four sides), a mix I created for my then-girlfriend in 1990 and a mix I created in the summer of 1992 for a woman in whom I was romantically interested. In the days before data compression and MP3s, constructing a mix required you to play every included track in its entirety. Plus, cassettes had only so much room and unless you bought them in bulk, blank ones were precious. Thus, you really had to like a track if you chose to record it onto a mix.

The brainstorm that I had in December 1992, then, was this: the universe of tracks included on my mixes roughly corresponded to my favorite tracks. And by tallying up what tracks appeared most often on a mix, I could both rank my favorite tracks and assign each one a numeric score. Then, by aggregating those scores, I could rank my favorite albums, artists and genres, both overall and by year.

Ranking hypothesis #1: I like every track I recorded onto a mix more than any track I have never recorded onto a mix.

Ranking hypothesis #2: The more mixes on which I have recorded a track, the more I like it.

This proved not to be as straightforward as I had hoped.


In March 1984, I drove some classmates and myself to Washington DC for a Model UN gathering. Anticipating the long drive, I made my first proper mix tapes since My Stuff: Georgetown Survival Mix Vol. I and II. We did not actually stay in Georgetown – where a teenaged Nell (now my wife) then lived – but in a Marriott on Connecticut Avenue NW in Woodley Park. These 43 tracks summarized favored recent album purchases: ABC’s The Lexicon of Love and Spandau Ballet’s True, plus an assortment of records from Roxy Music, Alan Parsons Project, Talking Heads, Genesis, Peter Gabriel, A Flock of Seagulls, U2, Duran Duran, Squeeze, Icehouse, Re-Flex and Real Life. I still did not have a proper cassette recorder – nor could I play cassettes in the “Berger Bus,” my black 1979 Ford Fairmont. So, my tape recorder came along for the ride.

Similar two-cassette “constructed” mixes followed in July 1984, November 1984 and January to March 1985 – with another “acquisition mix” in June 1984. That summer, as I settled into my new room in my mother’s suburban Philadelphia apartment – she had rented my bedroom to a young woman – I was gifted an all-in-one turntable/cassette player/radio/cassette recorder. I spent much of that summer twirling radio dials, seeking tracks to record, creating Summer 1985, Vols. I to VIII.

At the start of my sophomore year at Yale, after 28 mixes and 487+ unique tracks, I constructed a mix – mostly from my record collection, but with some borrowed albums as well – called Stuff and Such Vol. I. I have no idea why I chose this playful title, but in so doing I created the mix-naming convention I would use for the next 16 years.

Well, it was the naming convention I used exclusively, excepting Pseudo Dance Music Vols. I and II in October 1989, beginning with my move to the Boston suburbs at the end of August 1989. In the previous four years, I intertwined first 12 Stuff and Such volumes with Stuff of 1985, Summer 86 Vols. I-III, Summer 1987 Vols. I-III, Stuff of 1987-88, Video Stuff Vols. I-IV, Summer 1988 Vol. I and Washington Vol. I-III. Broadly speaking, I constructed Stuff and Such mixes from my record collection – filling out sides with a few radio/borrowed tracks – and the other mixes from the radio and elsewhere – filling out sides with tracks from my record collection.

Basically, mixes served two purposes at this time:

  1. Acquisition: Though borrowing and recording from the radio and cable music channels new tracks became part of my collection.
  2. Portability: I could play tracks from newly-acquired albums on my Sony Walkman anywhere I wanted.

By the end of August 1989, I had created 52 cassette and four video mixes – averaging seven per year – comprising 919+ unique tracks spread over 1,118+ “slots.” A “slot” is anything recorded onto a mix, so if I record 25 tracks onto a cassette, that is 25 “slots.” During this eight-year period, 157 tracks (17.1%) were recorded twice, while 21 (2.3%) were recorded three times. A track appears multiple times either because I liked it so much, I wanted it to include it on subsequent mixes, or because I first recorded one version (perhaps a 12” remix) from the radio then found a different version – or the video on either MTV or VH1.

Updated Ranking Hypothesis #2: The number of mixes onto which I record a track is positively associated with how much I like that track.

Meanwhile, as I prepared to move to Somerville to enroll in the Harvard Graduate School of Arts and Sciences doctoral program in government, I knew I had a long solo drive to Boston ahead of me. To accompany me on this drive, and presuming I could play these cassettes on the sound system of the U-Haul I drove, I lovingly prepared the 136-track Boston Drive Vols. I-VI. Only nine tracks were being recorded for the first time, including the first two tracks on Side 1 of Boston Drive Vol. I: Overture and Heaven on Their Minds, which open the soundtrack to the 1973 film version of Jesus Christ Superstar; I played that soundtrack incessantly over the next few years. This is the first instance of what I later called the “anchor” track (in this case, tracks) – the very first track recorded on a mix or set of mixes, the one(s) I am the most excited to hear. This would later apply to a) the first track recorded on any mix within a set and b) the final track recorded on that set.

Ranking Hypothesis #3: The tracks I like the most on a mix or set of mixes are the first tracks I record on a cassette/CD – especially the anchor track – and the final track I record.

Ranking Hypothesis #3a: RH3 is not true before August 1989.

Ranking Hypothesis #3b: RH3 is sometimes true between September 1989 and February 1992.

Ranking Hypothesis #3c: RH3 is always true after February 1992.

In essence, the Boston Drive mixes were a compendium of those tracks I always fast-forwarded or rewound to hear over the previous eight years. And for the first time, I began to think about how I ordered tracks. Up until now I had either been at the whim of disc/video jockeys or had recorded artist “blocks” – a group of ABC tracks followed by a group of Spandau Ballet tracks followed by…you get the idea. There was no particular ordering of tracks within each artist – or really at all.

This began to change with Boston Drive, even my thought process was no more complicated than “put ‘rocking’ tracks on Side 1 and ‘mellow’ tracks on Side 2,” with “rocking” and “mellow” loosely defined. This was how I constructed Vols. I-III. Side 1 of Boston Drive Vol. IV contains 10 Genesis tracks, while Side 2 opens with “The Chamber of 32 Doors” followed by seven instrumental tracks. Boston Drive Vols. V and VI are essentially the leftovers, with the line between “rocking” and “mellow” nearly obliterated. The final track – still one of my 10 or 15 favorites – is “Darkness,” the last track on The Police’s Ghost In The Machine.

The Boston Drive mixes were such a revelation I returned to them repeatedly over the next few years. They also allowed me essentially to ignore the previous 52 cassette mixes, though I still watched the four video cassettes sometimes; what I do not remember is whether I had physically left them behind in my mother’s suburban Philadelphia apartment.

Meanwhile, shortly after moving to Somerville, I borrowed an apartment-mate’s CD of Camper Van Beethoven’s Our Beloved Revolutionary Sweetheart. I had fallen in love with “One of These Days” after hearing it played on WFNX in Washington, though I never got the chance to record it. That track – still one of my 100 favorites – not only opened Stuff and Such Vol. XIII, it signaled the end of the first wave of mix making. I have not recorded a single track from the radio since then, for example, though I did once fill an entire videocassette from VH1 Classic, retroactively designating a set of videos as Video Stuff 2002-03. Also, I now focused solely on constructing cassette mixes based on a mix of newly-acquired tracks – whether from CDs, vinyl albums or cassettes, or culled from other collections – and “repeat” tracks, the ones I wanted to hear again after recording them on an earlier mix. I still grouped tracks by artist – devoting two cassettes to Genesis and one to Roxy Music – and gave little thought to how one track flowed into the next, but the first faint glimmers of the strict mix-construction rules I later followed are there.

This demarcation of mixes into “before Boston Drive” and “Boston Drive and later,” however, meant that when I began to enter track name, artist and mix name into my PC spreadsheet I began with the Boston Drive mixes. I was daunted enough by the thought of entering data from 32 mixes – 36 counting the tangential 80s and romantic mixes; entering data from the previous 56 mixes went above and beyond.

Moreover, I was not satisfied with a simple tally of how many of the 749 slots each of the 512 tracks filled; two-thirds (341) occupied only one slot, while 39 occupied three slots, six occupied four slots and five – “Zamba” by Bryan Ferry, “Save Me (plus the unlisted reprise after “Fat Chance Hotel” that closes out Happy?) by Public Image Ltd, “Same Old Scene” by Roxy Music, “Driver’s Seat” by Sniff ‘n’ the Tears and “Cuad El Habib” by Yello – occupied five slots.

No, I began to “weight” appearances on some mixes more than others. Nearly 30 years and many iterations later, I can only guess at those first weights – but an appearance on the Boston Drive mixes equated to something like three slots, while appearing on the four “non-series” mixes something like two slots, as did appearance on particularly beloved mixes like Stuff and Such Vol. XXX, created in June 1992. It is possible appearance on “one-artist-only” mixes counted as <1 slots, but I doubt it – I had not yet reached that level of, umm, methodological sophistication.

Ranking Hypothesis #4: I like tracks included on designated mixes more than those only included on non-designated mixes.

Ranking Hypothesis #4a: How much more I like tracks included on designated mixes varies by designated mix.

At the time, ignoring 56.1% of the 1,166+ tracks I had recorded over the previous 11+ years made logistical sense. It also allowed me to ignore the distinction between “now/recently” and “of all time.” I continued to exclude them – with a partial exception I revisit in the next installment – for more than a decade.

It was thus a limited collection of 512 tracks from which I tabulated – using the “ranking hypotheses” listed above – the first installment of “The Berger 100;”[3] “The Berger 10” received its own page:

10. “Promised You a Miracle”

9. “Are ‘Friends’ Electric?” Tubeway Army[4]

8. “The Evening’s Young” Yello

7. “New Toy” Lene Lovich[5]

6. “Stay Hungry” Talking Heads

5. “Cuad El Habib”

4. “Entangled” Genesis

3. “Right Down the Line” Gerry Rafferty

2. “Driver’s Seat”

1. “Save Me/Reprise”

“Zamba” was #12 and “Same Old Scene” was #19.

Based on what I remember of that time, the “ranking hypotheses” – the earliest incarnation of a score-computing algorithm – worked very well, at least in terms of what I most listened to in the early 1990s. From the song rankings, I generated “The Berger Album 50/10” and the “The Berger Artist 100/10.” At least, I think I did – I dated the track listings but not those for album and artist. Not that it mattered, as Ultravox’s Quartet and Genesis, respectively, topped those lists throughout the 1990s.

We return to that decade in the next installment, in which mix-making protocols emerge, mixes proliferate and technology flummoxes me…before it makes me rejoice.

Until next time…please be safe and healthy – and if you not already done so, please get vaccinated against COVID-19!

[1] I prefer “track” to “song” because it encompasses the full range of “music-related things that can be recorded onto a mix” tape, CD or video.

[2] For the previous two years, an older Harritonite rode the same bus as me, and she regularly played a mix of 60s folk rock tunes. The side I recorded – Beatles, Rolling Stones, Chicago, Seals and Crofts, Moody Blues, David Bowie, John Lennon – was inspired by her mix.

[3] Brian Eno’s “Julie With” was #100.

[4] Incorrectly written as Are Friends Electric? Gary Numan

[5] Yes, that is Thomas Dolby in the video.

How Likely Is Republican Control of the U.S. House In 2022?

On March 31, 2021, nearly five months after Election Day 2020, Democrat Rita Hart finally conceded to Republican Mariannette Miller-Meeks in Iowa’s 2nd Congressional District (“CD”), dropping her challenge to her six-vote loss. This was a net win for the Republicans, as United States House of Representatives (“House”) Member Dave Loebsack, a Democrat, had not sought reelection to an 8th term.

Overall, Democrats lost a net of 13 House seats relative to Election Day 2018, going from a 235-200 majority to a 222-213 majority. This double-digit seat loss occurred despite Democrats winning a 3.1-percentage-point (“point”) majority in all votes cast for U.S. House – 50.8 to 47.7%; notably, this was a 5.5-point decline from 2018

Returning to 2020, though, political observers were shocked at the number of House seats Democrats lost, especially because they won the presidency and netted three United States seats, giving them a 50-50 tie broken by Democratic Vice President Kamala Harris. Anyone following this website since at least June 2017, however, should not have been at all surprised.

Figure 1, using data from the 26 House elections from 1970 (vs. 1968) to 2020 (vs. 2018), shows that change in national House vote percentage accounts for an astonishing (for social science) 83% of the variance in House seat gain/loss. Reassuringly, the ordinary least squares (“OLS”) regression fitted to the data crosses the y-axis very close to the origin: a 0-point change in vote percentage essentially equates to no change in seats, which makes intuitive sense.

Figure 1: Change in % Democratic of Total House Vote vs. Net Change in Democratic House Seats, 1970-2020

Removing the 2018-20 data point from the OLS regression, suggests a drop in the Democratic share of the total House vote of 5.5 points yields an estimated loss of…

Dem Seat Change = 3.21 * (-5.5) – 1.19 = -18.8

Based on election data from the previous 50 years, Democrats should have lost 19 seats I 2020, with a 95% confidence interval (“CI”) of 18-20 seats. Yes, Democratic leaders expected to pick up seats, but history presents a compelling counter-narrative: Democrats were extremely lucky to maintain their House majority, however slender.


But what about 2022, the first midterm election of the Administration of Democrat Joseph R. Biden, Jr., when Republicans only need to net five seats to regain the majority? Figure 2 shows how poorly the first House elections after a new president is elected go for that president’s party – and getting worse over time. Granted, there have only been nine such elections starting with the 1962 midterms of Democratic President John F. Kennedy. Still, excepting the first post-9/11 midterm election – when Republicans under President George W. Bush in 2022 gained eight House seats – the party controlling the White House has lost an average of 53 House seats in the last 30 years. The overall average since 1962 is a net loss of 24 House seats…which would give Republicans a 237-198 edge going into 2023; the median is only -15, which would give Republicans a 228-207 edge.

Figure 2: 1st Midterm Seat Loss by Newly-Elected White House Party, 1962-2018

Based on the OLS regression equation in Figure 2, meanwhile, Democrats would be expected to lose 47 seats in 2022, dropping them to a 175-260 minority. However, because there are only nine data points, there is a great deal of “wobble” in this estimate – the 95% CI is a nonsensical -4,002 to +3,909.

In other words, all we really know from these nine 1st-term midterm elections is that anything between a loss of 61 seats and a gain of eight seats is historically plausible, with something in the 15-25 loss range most plausible.



This brings us full circle back to my model…and the question of what the margin in the total House vote will be in 2022. A little algebra reveals that to maintain a slender 218-217 House majority going into 2023, Democrats need to win the total House vote by at least 2.1 points – and likely at least 3.1 points (no net change) to account for a +/-1 seat 95% CI.

For comparison, political scientist Alan Abramowitz just published a model of net Democratic House seat gain. The two independent variables are 1) national House vote total (as opposed to change from previous election) and 2) number of House seats currently occupied by the president’s party. Those two variables account for 83% of the variance, as much as my single independent variable does. That said, Abramowitz’s model uses data from 1946-2018, giving him an additional 11 data points: 37 vs. 26. According to this model, Democrats would need to win the national House vote by about 6 points to maintain control, which equates to an increase of about 2.9 points from 2020.

Neither model is wrong, per sé. We are both trying to model a relatively infrequent occurrence while zeroing in a very precise outcome – the difference of a few seats – leading to the magnification of what really are small differences in outcomes. A 3-point lead and a 6-point lead are relatively close, and we agree Democrats have an uphill battle to retain the House, even without taking into account changes in CD lines due to reapportionment following the 2020 Census.

Technically, Abramowitz models Democratic lead on the “generic ballot question,” a variation of the poll question “If the election in your CD were held today, would you vote for the Democrat, the Republican, or some other party – or not vote at all.”

I recently compiled the results of the 20 generic ballot polls released publicly in 2021 then used the following steps to calculate a weighted-adjusted polling average (“WAPA”):

  1. Adjust raw margin – Democratic % minus Republican % – for “pollster bias,” as calculated for 538.com Pollster Ratings. Essentially, this is how much the pollster missed the final margin, on average, in recent polls of the same race.
  2. Average adjusted margins by how far the poll was conducted from Election Day 2022 – using midpoint of poll field dates – and 538.com pollster quality.
    • Time weight: (677 – days to Election Day)/677
    • Pollster weight: Numeric value of letter grade (A+ = 4.3, A = 4.0, etc.) divided by 4.3.

For example, the most recent generic ballot poll was conducted by Quinnipiac University from May 18 to May 24, 2021; it shows Democrats leading 50-41%. Quinnipiac has a historic Democratic skew of 0.5 points, meaning that 9.0-point lead is effectively an 8.5-point lead. The field midpoint was May 21, 2021, or 536 days until November 8, 2022, giving the poll a time weight of (677-536)/677 = 0.208. Their A- rating equates to a 3.7/4.3 = 0.860 pollster weight. Overall, this adjusted Democratic lead of 8.5 points has a weight of 0.208 * 0.860 = 0.179 – which, while not especially high, is highest among the 20 polls.

The raw average of these 20 polls is Dem+4.1, though mean Democratic bias of 0.5 points means this average is effectively Dem+3.6. Two polls without a 538.com pollster rating/bias were clearly conducted on behalf of Republicans, one in early April by PEM Management Corporation and one in late February by the National Republican Senatorial Committee. I traditionally used 1.5 as the bias for such polls, but I chose to match the most Democratic leaning poll in the 20: 3.5 for RMG Research. I also somewhat arbitrarily assigned these polls a letter grade of C; overall average was B-.

The initial WAPA I calculated was Dem+3.9, though it treated the multiple polls from RMG Research (2), McLaughlin & Associates (3) and Echelon Insights (4) as statistically-independent, even though polls conducted by the same firm are likely related. I thus calculated a second WAPA, which was the pollster-quality-weighted average of the bias-adjusted, time-weighted WAPA for each pollster. Basically, it is WAPA adjusted for pollster. This value was Dem+3.1

Averaging these WAPAs yields my best estimate of the November 2022 generic ballot as of June 2021: Dem+3.5. This is astonishing because it means support for Democratic House candidates increased 0.4 points since November – when, historically, support for the “out party” should be increasing.

If, in fact, Democrats win the national House vote by 3.5 points in November 2022, I estimate they would gain 0.3 seats, albeit with a 95% CI of -4.1 to +4.8 – which means they would have approximately a 94% chance of retaining the House. The Abramowitz model, however, estimates a 9-seat loss in this scenario – losing the House in the process.

It is not at all clear, of course, how well a 3.5-point lead in June 2021 translates to actual voting in November 2022. My poli-sci-sense suggests this lead – fairly robust since January, mind you – will slowly fade over the next year-plus due to a traditional complacency on the part of infrequent voters, in this case Democrats who may reason that as long as Biden is president, they do not need to vote in 2022, and a renewed enthusiasm on the part of out-party voters. This differential in voting enthusiasm, I suspect, is what leads to lopsided out-party victories in midterm elections.

Another reason for extreme caution is that this Dem+3.5 margin equates to Democrats 44.6%, Republicans 41.1%, Other/Undecided 14.3%. If Other earns the same 1.5% it did in 2020, that leaves fully 12.8% of voters up for grabs. It is not unreasonable – given voting enthusiasm differences – they split 2-1 for Republicans: roughly 8.5-4.3. This would actually give Republicans a national House vote lead of 49.6-48.9% on Election Day, a Democratic vote decrease of 3.8 points and a loss of 13 House seats (+/-2), very close to the recent median of -15.

Of course, if those undecided voters split evenly, Democrats are back to +3.5; a break toward Democrats seems extremely unlikely. So, let’s split the difference: a Democratic national House vote lead of 1.4 points, which equates to a loss of 6.3 seats, albeit with a 3.1 to 9.5 95% CI, giving Democrats something like a 15-20% chance of retaining their majority. This is as close to a “forecast” as I am willing to come in June 2021.

The bottom line is this: Republicans are favored to win back the House in 2022, though whether extremely narrowly or lopsidedly is far from clear. Historic trends in 1st midterm elections – of which there are only nine since 1960 – suggest Democrats could lose anywhere from 15 to 47 seats. Models with more data points – though still only 26-37 – suggest the shift is likely to be much smaller, anywhere from Democrats essentially holding serve – even netting a seat – to a Republican seat gain in the low double-digits.

Basically, keep an eye on the generic ballot numbers – if they stay close to Democrats ahead 3-4 points, they could be on the verge of defying decades of recent political history. If it drops closer to even, or Republicans pull slightly ahead – it will be a long night for House Democrats on Election Day 2022.

Until next time…please stay safe and healthy – and if you have not already been vaccinated against COVID-19, please do so!

Finding The Worst Character In Neo-Noir: Let The Voting Begin!

WARNING: Spoilers ahead!!

In two previous posts, I…

  1. Introduced two metrics, POINTS and Opportunity-Adjusted POINTS (“OAP”), to rank films by how often they are cited as “neo-noir,” allowing for how many reputable authors on film noir could have listed them.
  2. Selected 64 characters as contenders for “worst character in neo-noir.”

These 64 characters are evenly distributed across four loosely-defined categories: Corrupt Power, Crime Boss, Cunning Manipulator, Psychotic Loner/Hired Assassin.

Corrupt Power

Harry Angel (Angel Heart), Harry Callahan (Dirty Harry), Noah Cross (Chinatown), Tyler Derden (Fight Club), Judge Doom (Who Framed Roger Rabbit?), Tom Farrell (No Way Out), Lou Ford (The Killer Inside Me), Ras Al Ghul/Henri Ducard (Batman Begins) Alonzo Harris (Training Day), Mr. Hand (Dark City), Paul Kersey (Death Wish), Dr. Hannibal Lecter (Manhunter, The Silence of the Lambs), Charlie Meadows (Barton Fink), Captain Dudley Smith (L.A. Confidential), Stansfield (Leon: The Professional), Adrian Veidt/Ozymandias (Watchmen)

Crime Boss

Frank (Thief), Marv (Sin City), Frank Booth (Blue Velvet), Jack Carter (Get Carter), Alain Charnier (The French Connection), Francis Costello (The Departed), Lenny “Pluto” Franklyn (One False Move), Don Logan (Sexy Beast), Rick Masters (To Live and Die in L.A.), Neil McCauley (Heat), Liam “Leo” O’Bannon (Miller’s Crossing), Keyser Soze (The Usual Suspects), Tom Stall/Joey Cusack (A History of Violence), Marsellus Wallace (Pulp Fiction), The Joker (The Dark Knight), The Pin (Brick)

Cunning Manipulator

Catherine (Black Widow) Mike (House of Games), Jackie Brown (Jackie Brown), Suzanne Brown/Ann McCord (Red Rock West), Peter Cable (Klute), Lilly Dillon (The Grifters), Bridget Gregory (The Last Seduction), Andy Hanson (Before the Devil Knows Your Dead) Woo-Jin Lee (Oldeuboi), Terry Lennox (The Long Goodbye), Tom Ripley (The Talented Mr. Ripley, et al.), Leonard Shelby (Memento), Suzie Toller (Wild Things), Catherine Tramell (Basic Instinct), Mavis Wald (Marlowe), Matty Walker (Body Heat)

Psychotic Loner/Hired Assassin

Kevin (Sin City), Vincent (Collateral), Walker (Point Blank), Travis Bickle (Taxi Driver), Louis Bloom (Nightcrawler), Max Cady (Cape Fear), Anton Chigurh (No Country for Old Men), Jef Costello (Le Samourai), John Doe (Se7en), Alex Forrest (Fatal Attraction), Jame Gumb/Buffalo Bill (The Silence of the Lambs), Loren Visser (Blood Simple), Jules Winnfield (Pulp Fiction), Mr. Blonde/Vic Vega (Reservoir Dogs), The Driver (Drive, The Driver), Rorschach (Watchmen)

This is a (relatively) diverse and fascinating group. Ten of the 64 are women, though nine of them are in the Cunning Manipulator category, which speaks volume about gender roles (in both senses of the word) in neo-noir films. The one woman not so categorized, Alex Forrest, is killed off at the end of Fatal Attraction, even though her initial “crime” was asserting her own sexuality. Setting aside the ethnically-uncertain Ras Al Ghul, literal cartoon Judge Doom, non-terrestrial Mr. Hand and possibly-supernatural Charlie Meadows, there are six people of color, excluding Anton Chigurh, portrayed by Javier Bardem. Jack Carter and Don Logan are British, Alain Charnier and Jef Costello are French, Keyser Soze is…Hungarian, I believe…and Woo-Jin Lee is South Korean. The Pin and Suzie Toller are high school students—while Elijah Wood was just 24 when Sin City was released. Tom Ripley and Catherine Tramell are both LGBTQI+; Lilly Dillon has a very unusual relationship with her son, though not the one Noah Cross has with his daughter.

There are two characters each from Pulp Fiction, The Silence of the Lambs, Sin City and Watchmen. Robert DeNiro portrays three characters—Travis Bickle, Max Cady, Neil McCauley—while Mickey Rourke (Harry Angel, Marv) and Kevin Spacey (John Doe, Keyser Soze/Verbal Kint) each play two; Bickle and Cady both appear in films directed by Martin Scorsese. If you count the version of Dr. Hannibal Lecter in Manhunter, director Michael Mann is represented by four characters (Lecter, McCauley, Frank and Vincent), as are Ethan and Joel Coen (Chigurh, Meadows, Leo O’Bannon, Loren Visser) and Quentin Tarantino (Jackie Brown, Mr. Blonde/Vic Vega, Marsellus Wallace, Jules Winnfield). Christopher Nolan (Ras Al Ghul, The Joker, Leonard Shelby) and Scorsese (Bickle, Cady and Francis Costello) have three characters each; six other directors—John Dahl, Jonathan Demme, William Friedkin, David Fincher, Robert Rodriguez (with an assist from Tarantino and Frank Miller) and Zack Snyder have two characters each.


To mimic the ordering used by NCAA Basketball brackets, I used the product of POINTS and OAP to “seed” characters within each category from 1-16. Do not take these seeds too literally, as they reflect awareness of the film as a whole rather than the darkness of any specific character.

Figure 1: Worst Character in Neo-Noir, Initial Field of 64

I used the following rough criteria to determine “winners” in the first two rounds:

  • Whether the character gets away with her/his scheme—not necessarily the same as surviving, as John Doe shows.
  • The number of people that die at the character’s own hands
  • The number of despicable actions besides murder—raping your own daughter, as Noah Cross does, being the classic example
  • Intelligence: Suzie Toller may be a high school student but her IQ is well over genius level–and she is willing to pull out her own teeth to make her scheme work. This distinguishes characters who are “merely” brutal, like Marv or Mr. Blonde/Vic Vega.
  • What is the scope of the character’s villainy? Is it global—like Adrian Veidt’s plan to end the Cold War or Ras Al Ghul’s desire to “save” Gotham City—or is it more personal and banal—like Walker wanting his share of $93,000?
  • Does the character have a redemption arc?
  • Similarly, do we root for the character in some way? Motivation matters: Walker has no grand design beyond revenge and getting his money, Carter wants to avenge his brother, Brown wants to be free from Ordell Robbie, Tom Stall wants to forget his past life, Frank wants to settle down and have a family, and The Driver (in the 2011 film) wants to protect his new friends.
  • Is the character the nominal “hero” of the film? I discussed this in the previous post in reference to Harry Callahan, Paul Kersey, Frank, Walker and others.

With these very rough criteria in mind, we commence Round 1 of elimination.

Round 1

Corrupt Power

Noah Cross over Adrian Veidt. This was surprisingly tough. Cross is a brilliant and power-crazed man who rapes his own daughter—and walks away with his daughter/granddaughter after his daughter is shot by police officers. And Chinatown is the definitive neo-noir film. But “Ozymandias” murders people with his bare hands, is one of the most intelligent characters in cinema history and is willing to destroy New York City to end the alternate-timeline Cold War. And therein lies the rub…his motivation, however twisted, is just other-serving enough to eliminate him here.

Ras Al Ghul over Mr. Hand. The latter is an alien, full stop.

Alonzo Harris over Judge Doom. The latter is a cartoon character, full stop.

Harry Angel over Tyler Derden. Yes, the latter blows up entire buildings and convinces men to beat each other to a pulp—and sort of gets away with it. But Johnny Liebling literally sacrificed a random stranger to make a deal with the devil—and there is a reason the source novel was called Fallen Angel: Harry Angel is pure evil, with or without “Louis Cyphre” guiding him. Derden is also, you know, only a figment of The Narrator’s imagination.

Dr. Hannibal Lecter over Paul Kersey. As despicable as I think Kersey’s actions are, he is the nominal “hero” of Death Wish (and its many sequels), and he acts out of grief. Lecter is a sociopathic genius cannibal locked in a maximum security prison.

Stansfield over Harry Callahan. This is an upset, a 14 seed beating a 3 seed. But while Callahan may be “Dirty,” he is not a pill-popping DEA agent who would gleefully murder a 12-year-old girl in cold blood.

Tom Farrell over Lou Ford. Ford’s sociopathy is local, Farrell’s criminality is global.

Dudley Smith over Charlie Meadows. There is enough uncertainty over Meadows’ true nature—or how much of Barton Fink is in the title character’s mind—to eliminate him. Plus, L.A. Confidential is one of the premier neo-noirs—and the cruelly calculating Smith makes my skin crawl; his casual shooting of Jack Vincennes remains my greatest shock watching a film in the theater.

Crime Boss

Don Logan over Marv. This is the supreme upset—a 16 seed toppling a 1 seed—yet it was not a close decision. After re-watching Sin City, I realized that as criminal and violent as Marv is, he reserves his most extreme viciousness for the truly evil characters in Basin City: Kevin, in particular. We genuinely root for Marv; motivations matter. Logan, by contrast, terrifies even the most hardened criminals in Sexy Beast.

The Joker over Alain Charnier. Everyone remembers Heath Ledger’s Oscar-winning performance in The Dark Knight. I had to look up Charnier’s character’s name.

Frank Booth over Rick Masters. Masters is essentially an artist-turned counterfeiter who uses violence to protect himself in a mediocre movie. Booth is a drug-addled sociopathic sadist in a brilliant film who is among the worst movie villains ever.

Neil McCauley over Lenny “Pluto” Franklyn. This was a tough choice. I had forgotten about Pluto—the leader of the Los Angeles drug gang in the oft-overlooked One False Move. He is brilliant, patient and legitimately frightening. But McCauley simply operates at a completely different level. He plans intricate, massive-haul heists in broad daylight, and he is willing to abandon anyone at any time to save himself.

Jack Carter over Francis Costello. Two of the best gangster films ever made in Get Carter and The Departed. Two of the greatest actors of the last 75 years in Michael Caine and Jack Nicholson. Carter is someone we root for—he wants to avenge his brother—even as his violent depravity shocks us. Costello rules a vast criminal empire, untouched by the law, for decades. However, for all of Nicholson’s talent, Caine imbues Carter with an icy resolve that chills viewers…and, as I pointed out before, Costello is loosely based on “Whitey” Bulger.

Keyser Soze over The Pin. The Pin is a high school student, Keyser Soze…is Keyser Soze.

Tom Stall/Joey Cusack BARELY over Frank. A fascinating matchup between two very sympathetic—albeit violently criminal—men who just want to forget the past and be with their families. But their past won’t let them, so they must brutally destroy that past. The one difference is that we know Stall returns to his family, and his children (at least) welcome him. Frank’s ending is far more ambiguous.

Marsellus Wallace over Leo O’Bannon. Despite very little screen time, Wallace is the absolute dominant force in Pulp Fiction. Jules and Vincent work for him, the briefcase belongs to him (and, no, it is NOT his soul), Butch Coolidge is hiding from him and, well, there is that “medieval” thing. Not to take anything away from mob boss O’Bannon, but Miller’s Crossing is a long way from Los Angeles.

Cunning Manipulator

Matty Walker over Suzie Toller. I agonized the most over this decision, by far. Both of them get away with their crimes, perhaps ending up on the same tropical beach with the world thinking they are dead. Indeed, Toller does everything Walker does, with far more intelligence, dedication (she literally rips out her own tooth with a pair of pliers) and cool-headedness…and she is only a high school student. In the end, however, it boiled down to the “neo-noir” status of each character’s film. While I think Wild Things is very underrated, it simply is not the classic of neo-noir Body Heat is. For that reason, and for that reason alone, I extremely reluctantly chose Walker over Toller.

Catherine over Suzanne Brown/Ann McCord. The bottom line is this: Brown/McCord is not necessarily the worst villain in Red Rock West. Catherine is the only villain in Black Widow.

Lilly Dillon over Jackie Brown. Jackie Brown may be the most charming and delightful character on this list; I was pleasantly surprised how much I enjoyed Jackie Brown. By contrast, Dillon is…difficult to like.

Leonard Shelby over Andy Hanson. This is not close. Hanson arranges for his hapless brother to rob their parents’ jewelry store—no muss, no fuss, until the robbery goes horribly wrong. While he is an amoral jerk, Shelby lets himself become a serial killer rather than face the fact he is responsible for his wife’s death…assuming he still actually cares. He can always forget any despicable crime he commits, charming his way through life.

Peter Cable over Mavis Wald. These are two old-school, not especially interesting characters (1971, 1969) whose murders operate within a fairly narrow sphere. Cable was effectively a coin flip.

Tom Ripley over Terry Lennox. This is only a mild upset. Like Wald, Lennox is old-school; both emerge from classic Raymond Chandler novels. Ripley is also old-school, emerging from the brilliant mind of Patricia Highsmith. But Ripley keeps appearing in films, beginning with Plein soleil (Purple Noon) in 1960, and he is the poster-boy for manipulation, effortlessly becoming other people.

Catherine Tramell over Mike. This was a tough choice, as both are among the most skilled liars in all of neo-noir. However, Mike is primarily a phenomenally gifted con artist who only kills when absolutely necessary, and he does get defeated in House of Games. If I read the ending of Basic Instinct correctly, Tramell murders incessantly and gets away with it.

Woo-Jin Lee over Bridget Gregory. I admit to being at a disadvantage here: I have seen The Last Seduction twice, but I have not (yet) seen Oldeuboi. Still, here is what I do know. Gregory is driven by fear and revenge over her abusive husband Clay, played with slimy perfection by Bill Pullman. But she is not inherently bad; she mostly just wants to be left alone…though she allows an innocent man to pay the price for her crimes. Lee, by contrast, locks a man—admittedly no saint—in a room for 15 years, then maneuvers him into sleeping with his own daughter. The yuck factor alone propels Lee forward.

Psychotic Loner/Hired Assassin

Rorschach over Walker. For the second time, a 16 seed upsets a 1 seed. While Point Blank, along with Body Heat, Chinatown, L.A. Confidential and Taxi Driver, is of the five key neo-noir films—those with 20.0 POINTS or more—Walker is far too sympathetic to be a villain. He is left for dead at the film’s start, betrayed by his partners in crime. In fact, the entire film may be a revenge fantasy Walker plays out in his mind as he dies. Meanwhile, I suspect Watchmen, like Nightcrawler, will receive more recognition as a neo-noir over time. And Rorschach will take his place alongside Callahan, Kersey and others in the vigilante pantheon—though less sympathetic and more unsettling.

Vincent over Mr. Blonde/Vic Vega. Vincent is a meticulous planner, while Vega is a screw-loose thug.

Loren Visser over The Driver. As violent as the latter is, his redemption arc and the tenderness with which he moves Irene aside in the elevator before pummeling a hit man to death keeps him from advancing to the next round. Visser, for his part, is the textbook hired assassin: deadly, ruthless and unwavering.

Kevin over Max Cady. Cady is terrifying, almost animalistic in his single-minded quest for revenge. But Scorsese’s Cape Fear is a remake of a classic-era-ish film noir. And I have never felt a cold chill go up my spine like I did when I first saw Kevin appear in the doorway to Goldie’s bedroom, eyes hidden behind shiny glasses. Learning he is panther-like quiet, strong and fast—and a sadistic cannibalistic religious zealot—was my primary takeaway from Sin City. Elijah Wood has seriously dark depths.

Jef Costello over Alex Forrest. Forrest’s character gets a raw deal, full stop.

John Doe over Jame Gumb/Buffalo Bill. These are the twin nightmares of this category. In one corner is the unnamed serial killer who haunts the unnamed city of Se7en, dispensing divine retribution for violation of the seven deadly sins—even to the point of mutilating and punishing himself. In the other corner is the serial killer of The Silence of the Lambs who kidnaps, tortures and murders women to build a new skin for himself. The one key difference is that while both men die at the end of the film, Doe remains in control of the situation even after that. In fact, he is in control for the entire movie.

Anton Chigurh over Jules Winnfield. They are the yin and yang of hired assassins. Chigurh is quiet, patient and slavishly devoted to the toss of his coin. Winnfield is loud, impulsive and given to misquoting Biblical passages. Both are extremely effective, terrifying and survive the film. But Winnfield has a legitimate redemption arc, however incomplete—and he thwarts the coffee shop robbery.

Travis Bickle over Louis Bloom. I agonized over this match-up almost as much as Matty Walker versus Suzie Toller. This process began when I marveled at Jake Gyllenhall’s emaciated performance in Nightcrawler. Coincidentally, I noted the strong resemblance between these two lonely outsiders who prowl the night city, feeding off its dark criminality—and understanding that their perception is distorted, a half-view of reality. Both men survive at the end of the film, though while Bickle, despite being hailed as a “hero,” has not grown at all, Bloom now has a thriving, expanding video news production business. Two things elevate Bickle, however: Taxi Driver’s iconic status and the number of people he kills himself (Bloom does not directly kill anybody).


Moving on to Round 2

Corrupt Power

Noah Cross over Ras Al Ghul. Al Ghul genuinely thinks the League of Shadows are helping Gotham City by destroying it—and it is he who first trains Bruce Wayne. Compared to the narcissistic and greedy Cross, Al Ghul is downright sympathetic.

Harry Angel over Alonzo Harris. Harris is corrupt, but Angel borders on pure evil.

Dr. Hannibal Lecter over Stansfield. This was not as obvious as it might seem. In the context of Manhunter and The Silence of the Lambs, Lecter assists law enforcement in the pursuit of the Tooth Fairy and Buffalo Bill. Stansfield, by contrast, is the unequivocal villain of Leon: The Professional, ritualistically popping pills and psyching himself up with classical music. Leon, the hired assassin, is the sympathetic character. This should actually elevate Stansfield over Lecter. However, Lecter is only able to help Clarice Starling because he is locked in his cell, or masked and bound. He is a violent cannibalistic psychopath—WHO ENDS THE FILM ON THE LOOSE. Stansfield is blown up at the end of Leon.

Dudley Smith over Tom Farrell. Farrell is contemptible, a Russian spy embedded deep within the American government, but he is not the killer being sought in No Way Out. Smith is a cold-blooded killer, determined to erase anyone—allies and foes alike—who prevents him from seizing full control of organized crime in Los Angeles. Yes, he dies at the end of the film, but Farrell is captured, making it a wash.

Crime Boss

The Joker over Don Logan. Joker’s ability to strategize, his nihilism and his disinterest in material gain elevate him over the admittedly-petrifying loose cannon that is Logan.

Frank Booth over Neil McCauley. McCauley is a master criminal willing to cut social ties to save himself, but he is not inherently bad. Booth is.

Keyser Soze over Jack Carter. Carter is vicious, relentless and fear-inducing—but in the context of Get Carter, he is the hero: we want him to succeed. Soze makes other hardened criminals scared of their own shadows.

Marsellus Wallace over Tom Stall/Joey Cusack. After all of his deranged violence—violence he neither sought nor wanted—Stall has a final tender, wholly silent scene with the family he loves. When last we see Wallace, he is about to, you know, get medieval.

Cunning Manipulator

Catherine over Matty Walker. This is another upset, a 9 seed eliminating a 1 seed, though it was close. Walker goes through the machinations of killing her husband once, but Catherine does it at least three times. Walker gets away with her crimes, but Catherine is defeated. The difference is that Catherine is motivated by more than simple greed. She is a serial killer, titillated by the careful planning, and—unlike Walker—will keep being the black widow indefinitely.

Leonard Shelby over Lilly Dillon. What sets Shelby apart from Dillon, professional con artist and thief, is his willingness to “forget” all of his previous crimes. He chooses to be a serial killer because, like Catherine, some part of him enjoys it.

Tom Ripley over Peter Cable. Ripley is simply more devious and deviant—and vastly more interesting.

Catherine Tramell over Woo-Jin Lee. I nearly went the other way on this, but I know too little about Lee to be confident in my decision. And, reviewing the plot of Basic Instinct, Tramell is far more deadly and dangerous than I had recalled. Lee ruins one life—well, two—but Tramell kills early and often.

Psychotic Loner/Hired Assassin

Vincent over Rorschach. This also was not obvious. Vincent feels nothing for his fellow humans; Rorschach drips with contempt for them. Vincent kills because he is paid to do so, and his brilliance allows him to do so effectively and lucratively. Rorschach kills because he wants to clean society of its filth, and because he was severely traumatized as a child. But, despite being a nominal “superhero,” it is difficult to root for him. Yet, root for him we do—and we are genuinely upset when Dr. Manhattan kills him at the end of Watchmen. We are not remotely upset when Vincent dies at the end of Collateral.

Kevin over Loren Visser. The cannibal serial killer eliminates the hired assassin.

John Doe over Jef Costello. The zealot serial killer eliminates the hired assassin.

Anton Chigurh over Travis Bickle. Chigurh is evil at its most banal: indifferent, patient and calculating. But for his coin, he would kill many more people. Moreover, by OAP, No Country For Old Men is the post-1966 most often cited as “film noir.” Bickle, by contrast, is less evil than deeply troubled, unable to cope with his surroundings. He does not kill for money or sport, but to “cleanse” society by rescuing a single child prostitute. And he is the nominal “hero” of Taxi Driver.

And with that, the Not-So-Sweet Sixteen is set.

Figure 2: Worst Character in Neo-Noir, Not-So-Sweet 16

It is now time to vote on Twitter, so please find me there @drnoir33! I will keep early votes open longer, but not more than 36 hours or so.

Until next time…be safe and careful…and please get vaccinated!

The Not-So-Changing Geography of U.S. Elections

On November 3, 2020, Democrats Joe Biden and Kamala Harris were elected president and vice president, respectively, of the United States. According to data from Dave Leip’s essential Atlas of U.S. Presidential Elections, the Biden-Harris ticket won 51.3% of the nearly 158.6 million votes cast. Turnout shattered the previous record of 137.1 million votes cast in 2016: 15.6% more votes were cast for president in 2020 than in 2016. The incumbent Republican president and vice president, Donald Trump and Mike Pence, won 46.8% of the vote, with the remaining 2.0% going mostly to the Libertarian and Green tickets

While the 4.5 percentage point (“point”) margin for Biden-Harris over Trump-Pence—7.1 million votes—was solid, it is the Electoral College which determines the winner of presidential elections. Despite objections to the counting of the votes from individual states and an armed insurrection aimed to stop the Congressional certification of Electoral Votes (“EV”), the Biden-Harris ticket was awarded 306 EV—36 more than necessary—to 232 for Trump-Pence.

In many ways, the 2020 presidential election was a near-perfect encapsulation of recent presidential elections. Between 1992, when Democrats Bill Clinton and Al Gore were elected president and vice president, ending 40 years of Republican White House dominance, and 2016, the Democratic presidential ticket averaged a 3.6-point winning margin and 313.7 EV, very close to 4.5 points and 306 EV.

Biden-Harris improved on the 2016 Democratic margin in the national popular vote by 2.4 points, winning 16.4 million more votes than the ticket of Hillary Clinton and Tim Kaine; Trump-Pence won 11.3 million more votes, while third party candidates won 5.2 million fewer votes. Moreover, across the 50 states and the District of Columbia (“DC”), the Democratic ticket improved by an average of 3.1 points! In the EC, as Table 1 shows, Biden-Harris carried five states Clinton-Kaine lost in 2016: Arizona, Georgia, Michigan, Pennsylvania and Wisconsin; no states flipped the other direction.

Table 1: States with Presidential Election Margins <5.0 Points in 2016 and/or 2020

StateEV2016 Margin (Dem-Rep)2020 Margin (Dem-Rep)2020-2016
North Carolina15-3.7-173,315-1.3-74,483+2.4+98,832
New Hampshire40.4+2,7367.4+59,277+7.0+56,541

Clinton-Kaine won Virginia by 5.3 points in 2016; four years later Biden-Harris won the state by 10.1 points, a 4.8-point jump. The shift in Texas was similar, from a 9.0-point loss to “only” a 5.6-point loss, a 3.4-point improvement. In fact, Biden-Harris did better than Clinton-Kaine in every close state except Florida, losing by 258,775 votes more than in 2016. Overall, the only other states where the Democratic margin was at least 0.1 points worse in 2020 were Arkansas (-0.7), California (-0.8), Utah (-2.4) and Hawaii (-2.7). By contrast, Biden-Harris improved by at least 6.0 points (roughly double the state average) in the close states of Maine (6.1), New Hampshire (7.0) and Colorado (8.6), as well as Massachusetts (6.3), Connecticut (6.4), Maryland (6.8), Biden’s home state of Delaware (7.7) and Vermont (9.0).

Had Clinton-Kaine flipped just 77,736 votes in Michigan, Pennsylvania and Wisconsin in 2016, Democrats would have retained the White House, 278-260. By the same token, had Trump-Pence flipped just 65,009 votes in Arizona, Georgia, Wisconsin, and the 2nd Congressional District of Nebraska (“NE-2”), they would have been reelected, 270-268—while still losing the national popular vote by 4.5 points. Wisconsin, which shifted only 1.4 points—43,430 votes—toward the Democrats, was a key pivot state in both elections, with Pennsylvania right behind.


To better understand the relative partisan leans of each state, I developed 3W-RDM, a weighted average of how much more or less Democratic than the nation as a whole a state voted in the three most recent presidential elections. Basically, it is what I estimate the state-level margin between the Democratic and Republican nominees would be if they tied in the national popular vote. Note, however, that 3W-RDM (plus national popular vote) has missed the actual state-level result by an average of 5.3 points in recent elections. Figure 1 and Table 2 show current 3W-RDM for every state, based upon data from the 2012, 2016 and 2020 elections. Table 2 also lists 3W-RDM based upon data from 1984-92 and 2008-16.

Figure 1: Current State Partisan Lean, Based Upon 2012-20 Presidential Voting

Table 2: Current and Historic State Partisan Lean (3W-RDM), Sorted Most- to Least-Democratic

State2020 EV1984-922008-162012-20Ave. Change 1992-2020
New York2910.821.620.21.3
Rhode Island415.
New Jersey14-
New Mexico52.
New Hampshire4-
North Carolina15-7.0-6.0-5.80.2
South Carolina9-13.9-15.7-15.9-0.3
South Dakota3-5.5-25.8-29.6-3.4
North Dakota3-12.7-29.4-35.4-3.2
West Virginia59.2-35.5-41.4-7.2
AVERAGE -1.3-4.6-5.1-0.5

The core Democratic areas are primarily where they have been for 30 years: New England (average 3W-RDM: D+15.2), the Pacific Coast minus Alaska (D+12.4), the mid-Atlantic minus Pennsylvania (D+22). These 15 states and DC contain a total of 183 EV. Add the Midwestern states of Illinois (20 EV) and Minnesota (10), and the southwestern states of New Mexico (5) and Colorado (9), and the total rises to 226 or 227, depending upon Maine’s 2nd Congressional District (“ME-2”). This is the current Democratic presidential baseline, 44 EV from 270.

The core Republican areas are also primarily where they have been for 30 years: Mountain West plus Alaska minus Colorado (R+29.2); the six states running south from North Dakota to Texas (R+26.9); the five states in the western half of the Deep South (R+25.8); the border states of Missouri, Kentucky and West Virginia (R+30.3); and the Midwestern states of Iowa, Indiana and Ohio (R+13.1). Add the southern Atlantic states of North Carolina, South Carolina, Georgia and Florida, plus Arizona, and the total is 258 or 259 EV, depending upon NE-2. Each of these 27 states is at least 5.5 points more Republican than the nation, making it the current GOP presidential baseline, just 12 EV from 270.

Two states totaling 22 EV would be balanced on a knife’s edge: Michigan and Nevada. In 2016, they split, with Republicans winning the former and Democrats winning the latter. Biden-Harris won both in 2020.

That leaves two states totaling 30 EV—Pennsylvania (R+2.3) and Wisconsin (R+2.4); they lean more Republican than the “core” Democratic states of Minnesota and New Hampshire. Add them to the “core” Republican 258 EV, and Republicans enter a presidential race tied in the national vote—or even a point behind—with a minimum of 288 EV, 18 more than necessary. Michigan, Nevada, NE-2 and ME-2 would get them to 312.

I made this same point here, when I used a simple ordinary least squares (“OLS”) regression model of EV and national popular vote margin to show that in a dead-even national election, Republicans would—on average—be favored to win the EC 283-251, with four EV going to third-party tickets. Adding data from 2020 does not materially alter this estimate, which is essentially Republicans winning their 258 EV plus Pennsylvania and Wisconsin: 288 EV. Democrats then win their core states plus Michigan, Nevada, ME-2 and NE-2: 250 EV.

Here are the updated OLS regressions:

Democrats:               Electoral Votes = 1232.9*Popular Vote Margin + 250.98

Republicans:            Electoral Votes = 1229.2*Popular Vote Margin + 283.04

Simple algebra shows Democrats need to win nationally by 1.5 points to be on track to win 270 EV, while Republicans could lose nationally by 1.1 points and be on track to win. Put another way, Republicans could theoretically lose the national popular vote by 2.3 points and still win 288 EV, given the imbalance in the Electoral College.

Paradoxically, however, Democrats have won the EC in five of the last eight presidential elections, because they win the national popular vote by large enough margins. The 3.5-point average margin in those eight elections translates to an estimated 294 EV, on average: winning their core 226, plus Michigan, Nevada, ME-2, NE-2, Wisconsin, Pennsylvania (280 EV total) plus one of North Carolina, Arizona or Georgia. As we saw, the Biden-Harris ticket won all but ME-2 while adding Arizona and Georgia, losing North Carolina by just 1.3 points.

This imbalance has been getting worse over time. In the mid-1990s, after the Republican ticket won by landslides in 1984 and 1988 and Clinton-Gore won by a slightly smaller landslide in 1992, the average state was only 1.3 points more Republican than the nation, far lower than the roughly 5.0 points of recent elections. In a dead-even national election—essentially what happened in 2000—Democrats would have had a slightly higher base, ~230 EV from 18 states plus DC at least D+2.0, with the ~30 EV of Michigan, Connecticut, Maine and Delaware within 1.0 points either way. Democrats would start closer to 250 than 230 votes in this scenario, though there would still be ~275 EV from 27 states at least R+2.0; throw in Montana (R+1.6) and the total increases to 278. Still, Democrats were far closer to parity in the EC in the mid-1990s than they are now.

What changed?

Figure 2: Average Change in State Lean Since 1984-92

As Figure 2 clearly shows, the strength of state-level partisanship sharply increased over time: Democratic states become somewhat more Democratic, while Republican states became dramatically more Republican. Not only did the average state shift 3-4 points more Republican, relative to the nation, but the variance widened. After the 1984-92, the standard deviation—a measure of how narrowly or widely values are spread around the mean—increased from 14.4 to 23.4 after the 2012-20 elections. Moreover, consider states at least 3.5 points more partisan than the nation. In the mid-1990s, those states averaged D+12.8 and R+12.0; today, those values are D+19.5 (213 EV) and R+22.4 (259 EV).

The biggest pro-Democratic shifts, based upon the average three-election-cycle change in 3W-RDM since 1984-92, occurred in Vermont (average: D+3.2), the Pacific states of California and Hawaii (each D+2.7), and the mid-Atlantic states of Maryland, New Jersey, Virginia and Delaware and New York (mean: D+2.0). Colorado, Nevada, and the remaining New England states except Maine also shifted noticeably more Democratic. At Colorado, Nevada and Virginia even switched from core Republican states to core Democratic/swing.

But these shifts are miniscule compared to two blocks of Republican states. The first block I call the “upper interior Northwest”: Idaho, Montana, Wyoming and the Dakotas. These five states became an average 3.2 points more Republican every cycle since the mid-1990s. The second block I loosely call “Border,” though I could also call them “White, Culturally Conservative”: Oklahoma, Missouri, Arkansas, Tennessee, Kentucky and, most extremely, West Virginia. These six states became an average 5.3 points more Republican every cycle since the mid-1990s. West Virginia, in fact, is almost in a category by itself. Following the 1992 presidential election, when Clinton-Gore won it by 13.0 points, it has become an astonishing 7.2 points more Republican each cycle since then; Trump-Pence won it in 2020 by 38.9 points, a 51.9-point pro-Republican shift!

In fact, seven states—Arkansas, Kentucky, Oklahoma, South Dakota, Tennessee, West Virginia and Wyoming—shifted further Republican over 28 years than any state shifted Democratic over those years. West Virginia is also joined by Arkansas, Iowa, Missouri, Pennsylvania and Wisconsin as states that shifted from core Democratic to core Republican/pivot states.

As for why states shifted strongly Democratic or Republican, I wrote here about the growing partisan divide between white voters with (Democratic) or without (Republican) a college degree. Other explanations include self-sorting by geography (Democrats to the coasts, Republicans to “flyover” country) and information (Democrats from traditional media, CNN and MSNBC; Republicans from right-wing media and Fox News).


Thus far, I have only looked at presidential elections. Table 3 lists the percentages of United States Senators (“Senators”), Governors and Members of the United States House of Representatives (“House Members”) who are Democrats in the core Democratic, swing/pivot (Michigan, Nevada, Pennsylvania, Wisconsin) and core Republican states. Data on the partisan split of each House delegation, based upon the results of the 2020 elections, may be found here.

Table 3: Democratic Percentage of Senators, Governors and House Members in Three Groups of States

GroupSenatorsGovernorsHouse Members
Core Democratic (n=19)97.4%*78.9%76.9%
Swing/Pivot (n=4)75.0%100.0%50.0%
Core Republican (n=27)13.0%14.8%27.8%
* Includes two Independents, Angus King of Maine and Bernie Sanders of Vermont, who caucus with Democrats.

While not a perfect overlay, these percentages tell a simple story: states that lean Democratic at the presidential level strongly tend to elect Democrats to statewide office, while states that lean Republican at the presidential level strongly tend to elect Republicans to statewide office. Thus, only five of 57 (8.8%) Democratic-state Senators and Governors are Republicans: the indomitable Senator Susan Collins of Maine, and the governors of Maryland, Massachusetts, New Hampshire and Vermont. By the same token, only 11 of 81 (13.6%) Republican-state Senators and Governors are Democrats: all four Senators from Arizona and Georgia; one Senator each from Montana, Ohio and West Virginia (political-gravity-defying Joe Manchin); and the governors of Kansas, Kentucky, Louisiana and North Carolina. In other words, only 16 of 138 (11.6%) Senators and Governors from these 46 states are from the “opposition” party. Curiously, in the four swing/pivot states, every governor and Senator except Senators Pat Toomey of Pennsylvania and Ron Johnson of Wisconsin—the pivot states—are Democrats. The House percentages are a bit murkier, reflecting Republican pockets in “Democratic” states and Democratic pockets in “Republican” states, but it is still the case that roughly ¾ of the House delegations from these 46 states “match” their state’s partisan lean; swing/pivot states are split literally down the middle: 22 Democrats and 22 Republicans.

Pick your cliché. “All politics is local.” Clearly, not any more, as elections become increasingly nationalized. “I vote the person, not the party.” Apparently no longer true, given how closely voting for president/vice president, Senate, governor and House track. “Vote the bums out.” Well, voters seem to prefer bums from their party to anyone from the other party. As I noted with gerrymandering, these trends, if they continue, may be far more damaging for our two-party democracy than for either political party.

Until next time…please stay safe and healthy…

2020 Elections Post-Mortem

On November 3, 2020, the United States ended a weeks-long electoral process. At stake was the presidency, control of the United States Senate (“Senate”) and House of Representatives (“House”), 11 governor’s mansions, and thousands of state and local offices. That day, I published “cheat sheets” to guide election viewers through state-level presidential returns, 35 Senate elections and the gubernatorial elections.

[Ed. note: This post, my 200th, is the longest I have written to date. It is fitting that a blog which found its data-driven footing in the wake of the 2016 elections would have its 200th entry address the aftermath of the 2020 elections, beyond mere repetition of the number “20.”]

As I write this on midnight EST on November 17, 2020, precisely two weeks after the elections concluded, these are the top-line results:

  • Only one governor’s mansion changed partisan hands: Republican Greg Gianforte won back the statehouse in Montana for the first time in 16 years. As of January 2021, Republicans will hold 27 governor’s mansions, and Democrats will hold 23.
  • Democrats basically held serve in state legislative races. For more details, please see here.

On balance, the 2020 elections affirmed the status quo: a nation roughly evenly split between Democrats and Republicans, though it remains possible the former could control, however narrowly, the White House, Senate and House for the first time since 2010.

Presidential election

Biden-Harris are closing on 79.0 million votes (50.9%), shattering the previous record of 69.5 million votes won by Democrat Barack Obama and Biden in 2008. Trump-Pence have just under 73.3 million votes (47.3%), ranking them second in history. Biden has now appeared on three of the four presidential tickets to receive the most votes, with Obama-Biden earning 65.9 million votes in 2012, edging out Clinton-Kaine in 2016 by about 65,000 votes. Third party candidates are receiving more than 2.8 million votes (1.8%), significantly lower than the 8.3 million votes (6.0%) such candidates received in 2016. Approximately 155.1 million votes have already been counted, with an estimated 4.1 million votes—mostly in California and New York—left to be counted. This ~159.2 million vote projection, or about 2/3 of all Americans eligible to vote, also shatters the previous record of 137.1 million votes set in 2016.

Biden-Harris’ 3.6 percentage point (“point”) margin is a 1.5-point increase from 2016, and 0.3-point decrease from 2012, making it the third consecutive presidential election in which the Democratic ticket won the national popular vote by between two and four points; adding 22 million voters did not fundamentally alter the partisan electoral divide. Based on my Electoral College model, a Biden-Harris win of 3.6 points equates to 296 EV, nearly the 306 EV they received; for a Republican ticket, this equates to 327 EV.

How did Biden-Harris win the Electoral College?

Table 1: 2020 and 2016 Presidential Election Results by State, Ranked from Highest to Lowest Biden-Harris Margin

StateEVWinnerClinton-Kaine MarginBiden-Harris MarginDelta
Rhode Island4Biden15.520.85.2
New Jersey14Biden16.915.5-1.4
New York29Biden22.513.7-8.8
New Mexico5Biden8.210.82.6
Maine4Biden (3)
New Hampshire4Biden0.47.47.0
North Carolina15Trump-3.7-1.42.3
South Carolina9Trump-20.4-11.78.7
Nebraska5Trump (4)-17.8-19.2-1.4
South Dakota3Trump-29.8-26.23.6
North Dakota3Trump-35.7-33.42.4
West Virginia5Trump-41.7-39.02.7
Average  Trump+3.6Trump+0.8D+2.8

As Table 1 reveals, Biden-Harris won 25 states and the District of Columbia (“DC”) by an average of 17.4 points, while Trump-Pence won 25 states by an average of 19.8 points; medians are 14.6—reflecting the 86.8-point margin in DC—and 18.6, respectively. Biden-Harris won seven states and DC totaling 97 EV by 20 or more points, while Trump-Pence won 11 states totaling 65 EV by that margin.

Biden-Harris won 19 states, DC and the 2nd Congressional district in Nebraska by at least 6.0 points, for a total of 228 EV. Add Nevada (6) and Michigan (16), which the Democratic ticket won by ~2.5 points, below their national margin, and the total increases to 250 EV.

At around 10:30 am EST on Saturday, November 7, the major news networks declared Biden-Harris the projected winner in Pennsylvania—and its 20 EV put Biden-Harris over the total of 270 needed to win the presidency. It also makes Pennsylvania—the state in which I was born—the “tipping point” state, as it puts Biden-Harris over 270 EV when states are ranked from most to least Democratic. But the margin stands at just 1.0 points, or just 68,903 votes; Biden-Harris also won Wisconsin (0.6 points), Arizona and Georgia (0.3 points each) by similarly small margins. The Democratic ticket has a total winning margin of 104,025 votes in these four states.

In the 25 states, plus DC, won by the Democratic ticket, the average increase in margin from 2016 was 3.4 points, while in states won by the Republican ticket the average increase was 2.1 points; overall, the average margin shift was 2.8 points. In the five states which switched from Republican to Democratic, the average increase was 3.0 points, led by a 3.8-point increase in Arizona and a 5.4-point increase in Georgia. While Biden-Harris lost North Carolina by 1.4 points and Texas by 5.7 points, they improved the margin by 2.3 and 3.3 points, respectively.

However, while Biden-Harris improved on the 2016 margins by an average 3.7 points in these four southeastern/southwestern states—states I suggested were fertile ground for Democrats—they basically held serve in Iowa (D+1.2) and Ohio (no change), while falling further behind in Florida (D-2.2); I will not speculate what role undelivered ballots in Miami-Dade County played in the latter state. This should not be surprising, as these were perhaps the most disappointing states for Democrats during the otherwise “blue wave” 2018 midterm elections.

In 2016, Trump-Pence won 306 EV by winning six states Obama-Biden won in 2012: the aforementioned Florida, Iowa and Ohio, plus Michigan, Pennsylvania and Wisconsin. The latter were decided by a combined 77,736 votes; Clinton-Kaine also lost Georgia by 211,141 votes and Arizona by 91,234 votes. In 2020, as Table 2 shows, Biden-Harris won the former three states—more than enough to give them an Electoral College victory—by a combined 233,945 votes: a shift of 311,681 votes, or just 0.2% of all votes cast. But the Democratic ticket also increased their margin in Arizona by 101,691 votes and in Georgia by a remarkable 226,296 votes.

Table 2: Changes in Margin from 2016 to 2020 in Five Key States

State2016 Dem Margin2020 Dem MarginIncrease, 2016-20

Overall, across these five states, the margin swung toward the Democratic ticket by about 640,000 votes, which is still less than 1% of all votes cast. But we can get even more granular than that. Early in 2017, I observed that in the three states that swung the 2016 election to Trump-Pence, the Clinton-Kaine ticket did about as well in the Democratic core counties—the urban centers of Detroit, Milwaukee/Madison and Philadelphia/Pittsburgh—as Obama-Biden had in 2012. What changed was a massive increase in Republican turnout in the other, more rural counties of those states. I ultimately concluded this resulted from a split between white voters with a college degree (more Democratic suburban/urban) and without a college degree (more Republican rural).

Table 3: Changes in Margin from 2016 to 2020 in Pennsylvania Counties

County2016 D Margin2020 D MarginIncrease, 2016-20
Phila Suburbs   
Major Urban   
All Other Counties-816,051-826,874-10,283

Table 3 shows just how this split played out in 2020, using Pennsylvania as an example. Compared to 2016, the margins for the Democratic ticket increased only at 21,000 votes in the heavily urban Democratic counties of Allegheny (Pittsburgh) and Philadelphia. And the 61 counties outside these two counties, excepting the four-county ring around Philadelphia, also held serve for the Republican ticket; Erie and Northampton Counties switched, barely, from Trump-Pence to Biden-Harris. In fact, the two parties may have reach voted saturation in these two areas. But those four suburban Philadelphia counties, swung even further toward the Democratic ticket, from a margin of 188,353 votes to nearly 291,422 votes, for a total increase of 103,069 votes, nearly the entire swing from 2016 to 2020.

What about the polling?

With most of the vote counted, Biden-Harris lead Trump-Pence nationally by 3.6 points, which is 4.6 points lower than my final weighted-adjusted polling average (“WAPA”) of 8.2 points.

For my final post tracking national and state polling of the 2020 presidential election, I estimated the probability Biden-Harris would win a given state. In 24 states/DC totaling 279 EV, the probability was at least 94.7%; Biden-Harris won all of them. In 20 states totaling 126 EV, the probability was 1.3% or less; Trump-Pence won all them. The remaining seven states were:

  • Florida (80.1%), which Biden-Harris lost
  • Arizona (77.5%), which Biden-Harris won
  • North Carolina (69.0%), which Biden-Harris lost
  • Georgia (56.4%), which Biden-Harris won
  • Ohio (39.1%), Iowa (37.0%) and Texas (28.4%), each of which Trump-Pence won

Florida and North Carolina were the only “misses,” though it should be noted Trump-Pence still had a non-trivial 19.9% and 31.0% chance, respectively, to win those states. Further, my final back-of-the-envelope EV estimate was 348.5 for Biden-Harris—subtracting the 44 combined EV of Florida and North Carolina essentially gets you to 306. The latter value is also very close to the 297.5 EV I estimated Biden-Harris would receive if all polls overestimated Democratic strength by 3.0 points.

Along those lines, my 2020 election cheat sheets included a projected Democratic-minus-Republican margin (“JBWM”), which adjusts final WAPA for undecided votes, along with recent polling errors in selected states. Compared to the final FiveThirtyEight.com margins/polling averages (“538”), JBWM margins were about 1.2 points more Republican.

Even so, as Table 4 shows, the JBWM margins were, on average, 3.4 points more Democratic than the final margins, and the 538 margins were 4.6 points more Democratic. When the direction of the difference is ignored, meanwhile, the differences between the two method vanish: an average absolute difference of 4.5 from JBWM margins compared to 4.8 for 538.

However, this overall difference masks a stark partisan difference: the mean JBWM difference was only 1.1 points more Democratic in states/DC won by Biden-Harris, while it was 5.9 points more Democratic in states won by Trump-Pence. The correlation between the Biden-Harris margin and the JBWM difference is 0.73, meaning the more Republican the state, the better Trump-Pence did relative to the final polling. In short, pollsters continue to undercount “Trump Republicans” in the most Republican states.

Table 4: 2020 Presidential Election Results by State, Ranked by Difference from JBWM Democratic-Republican Margin “Projection”

StateEVWinnerJBWM ProjectionBiden-Harris MarginDelta
West Virginia5Trump-20.4-39.0-18.6
New York29Biden28.313.7-14.6
South Dakota3Trump-15.6-26.2-10.6
North Dakota3Trump-23.2-33.3-10.1
New Jersey14Biden19.515.5-4.0
Maine4Biden (3)12.58.7-3.8
South Carolina9Trump-8.9-11.7-2.8
New Hampshire4Biden8.97.4-1.5
Nebraska5Biden (4)-17.8-19.2-1.4
New Mexico5Biden12.010.8-1.2
North Carolina15Trump-1.1-1.4-0.3
Rhode Island4Biden19.020.71.7
Average  Biden+2.6Trump+0.8D-3.4

To again get more granular, Table 5 lists the pollsters who assessed the national popular vote at least five times since January 1, 2019, sorted by distance from the actual national margin of 3.6%. Margins are weighted for time, but not adjusted for partisan “bias.”

Table 5: Top 2020 Presidential Election Pollsters, Final WAPA National Margin

Pollster538 RatingFinal MarginDelta
NORC (AllAdults only)C+11.3-7.7
USC DornsifeB/C10.4-6.8
Quinnipiac UniversityB+10.4-6.8
NBC News/Wall Street JournalA-10.1-6.5
Global Strategy Group/GBAO (Navigator Res)C+9.9-6.3
Data for ProgressB-9.8-6.2
Redfield & Wilton StrategiesC+9.6-6.0
ABC News/Washington PostA+9.2-5.6
Marist CollegeA+9.1-5.5
Echelon InsightsC+8.8-5.2
Change ResearchC-8.3-4.7
Fox NewsA-8.3-4.7
Research Co.B-7.8-4.2
Morning ConsultB/C7.6-4.0
Monmouth UniversityA+7.4-3.8
Firehouse Strategies/OptimusB/C7.4-3.8
RMG ResearchB/C7.1-3.5
Harris XC6.5-2.9
Suffolk UniversityA6.2-2.6
Emerson CollegeA-3.8-0.2
Rasmussen Reports/Pulse Opinion ResearchC+3.20.4

           * John Zogby Strategies/EMI Solutions, Zogby Analytics, Zogby Interactive/JV Analytics

These 32 pollsters accounted for 556 (80.6%) of the 690 polls conducted. On average, they estimated Biden-Harris would win the national popular vote by 8.2 points, identical to my final WAPA; the average miss was 4.5 points in favor of Biden-Harris. There was only minimal difference by pollster quality: the 11 pollsters with a rating of B or better missed by an average of 4.2 points, while the 21 pollsters with a rating of B- or lower missed by an average of 4.7 points. That said, three of the four pollsters who came closest to the final national margin—Zogby, Rasmussen and Civiqs—had ratings of B/C or C+; the fourth was Emerson College, rated A-. At the other end of the spectrum are seven pollsters who anticipated a double-digit national popular vote win for Biden-Harris: low-rated Opinium, NORC (who polled adults, not registered/likely voters), CNN/SSRS, Qriously and USC Dornsife; and high-rated Quinnipiac University and NBC News/Wall Street Journal.

Overall, though, the polling captured the broad contours of the 2020 presidential election—if not the precise margins—fairly well, with JBWM and actual Democratic margins correlated a near-perfect 0.99; the order of states from most to least Democratic was accurately predicted. It forecast a solid, if not spectacular win by Biden-Harris in the national popular vote, a restoration of the upper Midwestern “blue wall,” and continued Democratic gains in southeastern/southwestern states such as Arizona, Georgia, North Carolina and Texas, even as Florida, Iowa and Ohio become more Republican.

One final note: it is exceedingly difficult to beat an elected incumbent president. Since 1952, it had happened only twice (1980, 1992) in eight chances prior to 2020[1]; Biden-Harris beat those 1:3 odds convincingly.

Senate elections

Democrats entered 2020 needing to flip a net four seats—or three seats plus the White House—to regain the majority for the first time since 2014. As Vice-President-elect, Kamala Harris breaks a 50-50 tie.

Table 6: 2020 Senate Election Results by State, Ranked from Highest to Lowest Democratic Margin, Compared to Pre-Election “Fundamentals”

StateWinnerFundamentalsFinal Dem MarginDelta
Rhode IslandReed24.433.08.6
New JerseyBooker18.415.8-2.6
New HampshireShaheen6.515.79.2
New MexicoLujan8.56.1-2.4
Georgia Special???-8.0-1.07.0
North CarolinaTillis-6.4-1.74.7
South CarolinaGraham-16.1-10.35.8
South DakotaRounds-26.2-31.5-5.3
West VirginiaCapito-35.9-43.3-7.4
AverageD+1 to 3GOP+8.9GOP+7.0D+1.9

Table 1 summarizes these elections; for the Georgia special election and Louisiana, margins are for all Democrats and all Republicans. Democrats John Hickenlooper and Mark Kelly defeated Republican incumbents in Colorado (Cory Gardner) and Arizona (Martha McSally), respectively, while Republican Tommy Tuberville defeated Democratic incumbent Doug Jones in Alabama. This leaves Democrats two seats shy of 50-50, pending the January 5 runoff elections in Georgia. Incumbent Republican David Perdue edged Ossoff on November 3 by 1.7 points, but fell 0.3 points short of the 50.0% needed to win outright. In the special election necessitated by the retirement of Republican Johnny Isaakson in December 2019, Warnock (32.9%) led incumbent Republican Kelly Loeffler (25.9%) by 7.0 points in the all-candidate “jungle primary;” overall, Republican candidates earned 49.4% of the vote and Democratic candidates earned 48.4%, with 2.2% split between a handful of third-party candidates.

When I took a “wicked early” look at these elections, I assessed the Democrat’s chance in each election using their “fundamentals,” or the sum of the state’s partisan lean (calculated using my 3W-RDM), the Democratic margin on the generic ballot and incumbency advantage.[2] For Table 6, the generic ballot is the difference in the percentages of the total vote for all Democratic House candidates and for all Republican House candidates; Democrats are ahead by 2.0 points.

On average, Democrats overperformed “expected” margins by 1.9 points. In the 13 elections won by Democrats, the overperformance was 2.5 points, while in the 20 elections won by Republicans, the overperformance was just 0.9 points; Democrats overperformed in the two Georgia Senate races by 7.0 and 8.3 points, confirming how rapidly it is moving toward swing-state status. The biggest Democratic overperformance—fully 10.7 points—was in Arizona, which in 2021 will have two Democratic Senators (both of whom beat McSally) for the first time since 1953. Other Senate elections in which the Democratic candidate overperformed by at least 9.0 points were New Hampshire, and three states where Democrats fell short in their attempt to win back a Republican-held seat: Montana, Kansas and Kentucky.

On the flip side, setting aside a 15.1-point underperfomance in Nebraska, the biggest Republican overperformance was in Maine, where incumbent Susan Collins, first elected in 1996, “should” have lost by 5.5 points. Instead, she won by 8.9 points; this is a 28-point decline from 2014, when Collins won by 37 points. Pending the results of the Georgia runoff elections, Maine is the only state in 2020 to have a Democratic presidential victory and a Republican Senate victory, with a gap of 17.6 points. It will be interesting to see whether Collins adjusts her voting in the next Senate. Other large Democratic underperformances, finally, took place in Michigan, where first-term Democratic Senator Gary Peters beat Republican John James by only 1.5 points and in West Virginia, which grows more Republican every year.

On the whole, though, expected and actual margins aligned nearly perfectly, with a 0.94 correlation.

What about the polling?

As with the presidential election, the final polling averages/projected margins were far less accurate, as Table 7 shows; I did not calculate a projected final margin for the Louisiana Senate election.

Table 7: 2020 Senate Election Results by State, Ranked by Difference from JBWM Democratic-Republican Margin “Projection”

StateWinnerJBWM ProjectionDemocratic MarginDelta
West VirginiaCapito-20.6-43.3-22.7
South DakotaRounds-19.9-31.5-11.6
New JerseyBooker24.615.8-8.8
South CarolinaGraham-4.7-10.3-5.6
New MexicoLujan10.06.1-3.9
North CarolinaTillis1.1-1.7-2.8
New HampshireShaheen14.415.71.3
Georgia Special???-3.9-1.02.9
Rhode IslandReed29.633.03.4
AverageDem+1 to 3GOP+0.6GOP+6.4D-5.8

The polling may have been within historic parameters for the presidential election, but it was far worse in the Senate elections, with the JBWM margins overestimating Democratic margins by an average of 5.8 points, almost exactly the 6.0 points by which 538 margins erred on average; ignoring direction, the average misses are 6.3 and 7.0 points, respectively. That said, the correlation between the actual and projected Democratic margins was 0.97, meaning the polling correctly forecast the order of Senate elections from most to least Democratic.

These overall averages again mask substantial partisan differences. In the 13 states where the Democratic nominee won, the average miss was a historically-reasonable -2.9 points, but in the 19 states (excluding Louisiana) where the Republican nominee won, the average miss was an astounding -8.3 points. Put another way, in the 15 states Trump-Pence won by at least 10 points which also held a Senate election, the average Senate miss was -8.9 points, while it was -3.3 points in all other states. Somewhat reassuringly, in the five states whose presidential margin was within five points also holding a Senate election (Arizona, Georgia, Michigan, North Carolina), the miss was only -2.2 points. Overall, the correlation between the Biden-Harris margin and the JBWM margin error was 0.57, confirming the idea pollsters systematically undercounted Republican support in the most Republican states.

My back-of-the-envelope estimate was a net gain of five Democratic seats in the Senate, with at least a 77% chance Democrats would regain control; these values dropped to 30% and either two or three seats with the assumption all polls systematically overestimated Democratic strength by three points. Democrats will ultimately net between one and three seats, corresponding more with the latter assumption. I estimate the probability Democrats win both Georgia Senate runoff elections—and thus the Senate—is between 25 and 50%, depending on the degree of ticket-splitting.

From a purely mathematical perspective, the largest Democratic underperformances occurred in the Senate elections in West Virginia, Wyoming, South Dakota and Nebraska: four extremely Republican states. But from a strategic perspective, the most disappointing elections were in Maine (-12.2) and North Carolina (-2.8), where incumbent Republican Thom Tillis narrowly held off a challenge from Democrat Cal Cunningham, who may have been hurt by a sexting scandal; given the narrowness of his victory (1.7 points) and the increasingly swing status of North Carolina, Tillis’ voting patterns also merit watching. These were the two states besides Arizona (98.1%) and Colorado (99.5%) in which I estimated the Democratic nominee had at least an 85% chance to defeat a Republican incumbent; I also thought Democrat Theresa Greenfield was roughly even money to defeat incumbent Republican Joni Ernst, despite projecting a final margin of 3.6 points for Ernst; the latter won by 6.6 points.

There were four additional Senate elections—in Alaska, Kansas (open seat), Montana and South Carolina—where I estimated the probability of a Democratic flip was between 11.7 and 26.4%. In a sign of how good these elections were for Republicans, their nominees won all four elections by an average of 11.3 points, a mean 7.5 points more Republican than projected. In fairness, these states tilted an average 19.2 points more Republican than the nation as a whole coming into the 2020 elections. A similar story can be told in Texas, which tilted 15.3 points more Republican, but where Democrat M.J. Hegar “only” lost by 9.8 points to incumbent Republican John Cornyn, beating expectations by 0.6 points.

Put simply, assuming a loss in Alabama, Democratic hopes of winning back control of the Senate relied on flipping two Senate seats in Democratic states, then winning at least two more seats in states ranging from somewhat Republican—Iowa, North Carolina, Arizona, Georgia—to extremely Republican—Alaska, Kansas, Montana, South Carolina and Texas—all while Trump sought reelection. To date, Democrats have only flipped seats in Colorado (D+2.2) and Arizona (D-9.7) while winning back the Vice-Presidency, losing tough elections in Iowa, Maine and North Carolina, while never really being in contention anywhere else. Senate control now rests on Democrats winning two Senate runoff elections in a nominally Republican state (D-9.6), but one where Biden-Harris won, improving on Clinton-Kaine’s by 5.4 points.

Gubernatorial elections

Unlike those for the White House and Senate, there was very little drama in these elections. Two Democratic incumbents—John Carney of Delaware and Jay Inslee of Washington—were expected to win easily; they won by margins of 20.9 and 13.6 points, respectively. Six Republican incumbents—Eric Holcomb of Indiana, Mike Parson of Missouri, Chris Sununu of New Hampshire, Doug Burgum of North Dakota, Phil Scott of Vermont and Jim Justice of West Virginia—as well as Republican Spencer Cox of Utah were expected to win easily, though I projected Parson to win by “only” 8.0 points (he won by 16.6 points). They won their elections by an average margin of 31.6 points!

The only possible drama was in Montana, where Republican Gianforte and Democrat Mike Cooney vied to win the governor’s mansion being vacated by Democrat Bullock, and North Carolina, where Democratic Governor Roy Cooper—who won extremely narrowly in 2016—faced Republican Dan Forest. Gianforte defeated Cooney by 12.4 points, easily exceeding a projected 4.5 points, while Cooper won by 4.5 points, not the projected 10.4 points. Still, my global projection was correct: a net gain of one governor’s mansion by the Republicans, giving them a 27-23 majority; this an overall net gain of seven governor’s mansions by the Democrats since 2016.

In these elections, Republicans strongly overperformed fundamentals (7.1 points) and JBWM projections (7.6 points). However, expected values were strongly skewed by Scott’s 41.1-point victory in extremely-Democratic Vermont (D+27.7) and Sununu’s 31.8-point victory in swing New Hampshire (D+0.1); exclude those two margins and DEMOCRATS overperformed expectations by 1.0 points—with Democrat Ben Salango exceeding what were admittedly very low expectations by 8.5 points. Meanwhile, in the four states with governor’s races won by Biden-Harris, Democratic gubernatorial nominees finished an average 8.9 points lower than projected, while in the seven states won by Trump-Pence, they finished an average 6.8 points worse than expected. Once again, the extreme disparity in presidential/Senate and gubernatorial voting in New Hampshire and Vermont—two of three states in solidly-Democratic New England, along with Massachusetts (Charlie Baker), to have very popular Republican governors. In fact, gubernatorial elections are among the only ones in which ticket-splitting is still relatively common: Biden-Harris won six states with a Republican governor,[3] while Trump-Pence won five states with a Democratic governor.[4]

House elections

A wide range of forecasters expected Democrats to net between five and 10 House seats[5]. I was highly dubious of this, to be honest, given the likelihood the margin for Democrats in the total national House vote would decline from the 8.6-point margin they earned in 2018; it would also be higher than the 1.1 points by which they lost in 2016, when they still managed to net six seats. However, because I was not closely tracking House races, I said nothing about my doubts.

According to the Cook House vote tracker, Democrats had earned more than 75.1 million House votes (50.1%), Republicans had earned just under 72.1 million votes (48.0%), with the nearly 2.2 million votes (1.8%) going to third-party candidates. A total of 150.0 million votes have been counted, 5.1 million less than those cast in the presidential election. The 2.0-point margin by which Democrats are winning the House vote—just under 3.1 million votes—is also lower than the 3.6 points, and 5.6 million votes by which Biden-Harris currently lead Trump-Pence. It is also much lower than the 9.7-million Democratic vote margin in 2018, albeit with 36.3 million more votes cast in 2020, reinforcing the conclusion a few million Republican-leaning voters “balanced” a vote for Biden-Harris with Republican votes elsewhere…or simply chose not to vote in down-ballot elections.

In the races that have already called, Republicans have gained 11 seats held by Democrats (two each in California and Florida, one each in Iowa, Minnesota, New Mexico, New York, Oklahoma, South Carolina and Utah), while Democrats have gained three seats held by Republicans (two in North Carolina, one in Georgia). This gives Democrats 221 seats, three more than needed for the majority, and Republicans 208 seats. Of the six seats yet to be called, Democrats currently hold four, with freshman Democrat Tom Malinowski leading by ~5,000 votes in New Jersey’s 7th Congressional District (“CD”). Giving that seat to the Democrats—and giving Republicans their open seat in New York’s 2nd CD—increases the totals to 222 Democrats and 209 Republicans.

That leaves four seats truly in doubt:

  • California’s 21st CD, where incumbent Democrat T.J. Cox trails Republican David Valadao, in a 2018 rematch, by 2,065 votes.
  • California’s 25th CD, where Democrat Christy Smith is within 104 votes of unseating Republican Mike Garcia, who won a special election in May 2020 after first-term Democrat Katie Hill resigned.
  • Iowa’s 2nd CD, where Democrat Dave Loebsack did not seek reelection; Republican Mariannette Miller-Meeks leads Democrat Rita Hart by only 47 votes!
  • New York’s 22nd CD, where Republican Claudia Tenney’s lead over incumbent Democrat Anthony Brindisi continues to shrink as New York votes are slowly counted.

Democrats will thus lose a net 8-12 seats compared to the 234-201 margin they had after the 2018 elections. This is a bad result for the Democrats, right?

Well, no…it suggests that polling-based expectations were flawed, because the fundamentals always pointed toward a net loss of House seats for the Democrats. Moreover, the comparison should be to 2016, because that is the last election in which Trump appeared on the ballot.

Following the 2016 elections, Republicans had a 241-194 House majority. Democrats were convinced, wrongly I thought, that gerrymandering by Republican legislators and governors would keep them in the minority for the foreseeable future. Looking ahead to the 2018 midterm elections, knowing Democrats needed to net 24 seats to regain the majority, I looked at all House elections from 1968 to 2016, and I noticed that what “predicted” net change in seats from one election to the next was not the national margin in a given election, but the change in that margin from the previous election. Figure 1 helps to illustrate this.

Figure 1:

In 2018, Democrats net a surprisingly-high 41 House seats, 17 more than they needed, most by narrow margins. It is then reasonable to expect that even a small decline in the Democratic share of the total national House vote would allow Republicans to “claw back” some of these seats Democrats currently lead the total national House vote by 2.0 points, fully 6.6-point decrease f 2018. Entering this value into the OLS regression shown in Figure 1 yields an estimated Democratic loss of 22.4 seats.

In other words, while Democrats expected to gain seats—based on what we now know was polling that underestimated Republican margins by 3-7 points—they should actually have been bracing themselves for a possible loss of the House itself. Instead, they “only” lost between eight and 12 seats, meaning they did far better than history would have suggested. Moreover, Democrats have net between 29 and 33 seats since 2016, earning control of the House in back-to-back elections for the first time since 2006-2008, something that seemed nearly impossible early in 2017.


Both Democrats and Republicans can find 2020 election results to celebrate.

Democrats won back the White House after just four years (beating 1:3 odds to defeat an incumbent), rebuilding their upper-Midwestern blue wall while expanding into the southeast and southwest; no Democratic presidential nominee has won both Arizona and Georgia since 1948. They also maintained control of the House of Representatives and made gains in the Senate; with two more wins in Georgia in January 2021, they regain control of the Senate as well. Democrats have not controlled both the White House and House since 2010.

Republicans, even as they lost the White House, gained as many as 12 seats in the House and staved off losing control of the Senate until January 2021 at the earliest. They net one governor’s mansion, giving them a 27-23 majority, and held their own in state legislative elections. Once again, Trump’s name on the ballot encouraged many more exurban and rural voters to vote than expected, ironically helping all Republicans but himself and his running mate.

Fans of bipartisan “balance” can also celebrate 11 states seeing different parties win their state’s electoral votes and serving as governor. Moreover, a record-smashing 155.1 million—and counting—Americans cast a ballot for president, which equates to two in three of all adults eligible to vote.

Finally, the polls erred substantially in favor of Republicans, with a miss of around 3.5 points compared to my final projections and 4.7 points relative to those from 538. Republicans fared even better in Senate and gubernatorial elections, beating final projections by around six points in the former and nearly eight points in the latter. These values mask a partisan split, with polls far more accurate for Democratic candidates than Republican ones. In the end, though, polls were far less accurate—in this Trump-led cycle at least—than simply considering a state’s recent partisan lean, the national partisan environment and incumbency. These fundamentals remain extremely predictive, at least relatively.

Until next time…please stay safe and healthy…

[1] 1956, 1972, 1980, 1984, 1992, 1996, 2004, 2012

[2] Democratic full-term incumbents=4.4, Democratic partial-term incumbents=2.2, non-incumbent=0, Republican partial-term incumbents=–0.4, -0.6, -1.6; Republican full-term incumbents=-2.4

[3] Arizona, Georgia, Maryland, Massachusetts, New Hampshire, Vermont

[4] Kansas, Kentucky, Louisiana, Montana, North Carolina

[5] The Cook Political Report hedged a bit, labeling 229 seats at least Lean Democrat, 179 seats at least Lean Republican, and 27 seats Toss-up. Of the Toss-ups, nine are held by Democrats, 17 by Republicans, and one by Justin Amash of Michigan, who switched from Republican to Independent in July 2019.