About those recent presidential approval polls…caveat emptor

During President Donald Trump’s recent combative press conference, he cited a new Rasmussen poll showing him at 55% approval.

What the…? I thought.

At first, I thought he had simply read the “disapprove” number as the “approve” number, because all of the presidential approval numbers I had been hearing (primarily from Gallup tracking polls) showed Trump’s approval percentage below 50%.

But then I checked the presidential approval polls on Huffington Post Pollster. Sure enough, the latest Rasmussen poll showed Trump at 55% approval and 45% disapproval, for a net approval of +10%. This poll of 1,500 likely voters was conducted from February 13 to 15, 2017.

My memory of the Gallup polls was also correct, however. The most recent Gallup survey (1,500 adults, February 11-13, 2017) showed Trump with 40% approval and 54% disapproval, for a net approval of -14%.

Umm, excuse me?

Since Trump’s inauguration on January 20, 2017, Gallup and Rasmussen have been surveying Trump’s approval rating using rolling three-day samples; Rasmussen presented four-day samples twice. That means that they began by surveying 500 people each day for three days to obtain an initial sample of 1,500 people. On the fourth day, they replaced the first day’s 500 respondents with a new sample of 500 respondents. This means that the sample of 1,500 respondents is completely updated every three days.

Besides the sampling universe (all adults for Gallup, likely voters for Rasmussen), another difference between the two polling organizations is that Gallup uses live telephone interviewers, while Rasmussen uses Interactive Voice Response (IVR—essentially automated telephone calls) supplemented by an online survey tool. Also, while Gallup allows respondents to provide an answer other than “approve” or “disapprove” (e.g., I don’t know), Rasmussen forces respondents to choose one of these two responses, and thus their percentages always sum to 100.

This was not the first time I had observed substantive differences between Gallup and Rasmussen presidential approval polls. In April 2016, I collected the 100 most recent polls on then-President Barack Obama’s approval (midpoints: January 27 to April 5, 2016). Overall, Obama’s averaged 48.1% approve and 48.4% disapprove, for a net approval of -0.3%. The 22 polls from Gallup, however, averaged 49.5% approve and 46.6% disapprove. The 16 Rasmussen polls averaged 49.2% approve and 49.9% disapprove; the average 0.9% neither approving nor disapproving likely results from rounding reported values to zero decimal places.

Thus, early in 2016, Gallup showed Obama’s net approval at +2.9% while Rasmussen pegged it at -0.7%, a not insubstantial difference of 3.6 percentage points.

What about in the first four weeks of the Trump Administration?

Overall, 40 non-overlapping (i.e., completely updated three-day samples, as appropriate) presidential approval polls have been conducted since January 20, 2017 by 13 different organizations[1]. President Trump has averaged 45.6% approval and 47.6% disapproval over this period, for a net approval of -2.0%. Examining values by consecutive five-day periods, however, shows a declining net approval, from +4.0% to -1.7% to -2.5% to -5.6% to -2.5%.

By contrast, the averages in the nine Gallup presidential approval polls were 43.0% approve/50.4% disapprove, for a net approval of -7.4%, while those in the seven Rasmussen presidential approval polls were 54.3% approve/45.7% disapprove, for a net approval of +8.6%!

In other words, two polling agencies, surveying American adults over the same four week period, found an astonishing 16.0 percentage point difference in President Trump’s net approval rating!

And the gap appears to be widening, especially over the last week, according to Figure 1 below:

Figure 1: Net Approval for President Donald Trump (% Approve – % Disapprove) in Post-Inauguration Polls

trump-net-approval-post-inauguration

Dates shown in Figure 1 are the survey midpoint (or the day after, if the survey was conducted over an even number of days). If multiple surveys had the same midpoint, their values were averaged.

Besides the 16 total Gallup and Rasmussen presidential approval polls, Figure 1 also displays the other 13 presidential approval polls that sampled all adults (average net Trump approval -4.5%, average sample size 2,903), the seven that sampled registered voters (+0.9%, 1,600) and the other four that sample likely voters (-0.2%, 835).

There is admittedly a lot of noise in these data, especially for the 24 polls not from Gallup or Rasmussen. Still, the stark contrast between the Gallup and Rasmussen presidential approval polls is evident, as is the fact that the Gallup net approval numbers are clearly declining. It is less clear whether the same is true for the Rasmussen net approval numbers.

This large difference between the presidential approval polls conducted in the last four weeks by Gallup and Rasmussen may simply result from differences in sample populations. Excluding Gallup and Rasmussen’s presidential approval polls, the difference in average net approval between polls conducted on all adults and those conducted only on likely voters is 4.3 percentage points. And while this gap is substantive, it pales next to the difference in average net approval of 10.9 percentage points when the Gallup and Rasmussen polls ARE included, and it really pales next to the 16.0 percentage point difference discussed earlier.

The remaining difference could be explained by interviewing strategies. Gallup uses live interviewers, who dial telephones and speak directly to respondents, while Rasmussen uses automated (forced-choice) and online techniques. There is a compelling notion that respondents will give more socially “acceptable” answers to a live interviewer than they would to an anonymous machine, but now that Trump is actually president, I suspect the level of “social unacceptability” has declined, if it ever existed.

Basically, differences in sample populations and interviewing strategies account for much of the disparity between the presidential approval polls conducted by Gallup and Rasmussen, but not all of it.

For now, the best course, as always, is to examine an aggregate of presidential approval polls, which suggest a net approval of about -4.0% over the last week (-5.9% on the latest Pollster chart)—which, while not in double-digit Gallup territory, is still lower than any recent president only one month into his first term.

Until next time…

[1] Besides Gallup and Rasmussen, there were four presidential approval polls each from Ipsos/Reuters, Politico/Morning Consult and YouGov/Economist; three from PPP; two each from Quinnipiac and Survey Monkey; and one each from CBS, CNN, Fox, Pew and Zogby. The averages across these 24 surveys were 44.3% approve, 46.9% disapprove (8.8% neither!), for a net approval of -2.6%.