“Experiment is the sole source of truth. It alone can teach us something new; it alone can give us certainty.”

Henri Poincaré

Politico, Adam Schaeffer: GOP needs a scientific revolution

image

GOP needs a scientific revolution

Politico, Adam Schaeffer
March 19, 2013 by Adam Schaeffer
[Link to Original]

Democratic pollster Pat Caddell touched off a prairie fire of grassroots rage at last week’s Conservative Political Action Conference with a passionate speech claiming widespread corruption in the ranks of Republican consultants. Rush Limbaugh fanned the flames, arguing that establishment players are ideologically soft, and more interested in approval from the D.C. cocktail set and making money than in political victory.

These bald accusations are the most aggressive from a growing chorus critical of the performance of the Romney campaign, the Republican Party and conservative organizations. Critics argue that the Romney campaign’s polling, messaging, strategy, tactics, and targeting were all failures. And, as Caddell implied, they were so obvious and complete a failure as to indicate not just incompetence but malfeasance on the part of those in charge.

Unfortunately, we are perilously close to turning what could and should be a revitalizing self-examination into a witch-hunt largely based on a conventional wisdom, which concludes everything a losing campaign did was wrong and everything a winning campaign did was correct. But is there any doubt that most pundits would be praising the very people being condemned had some outside event intervened to elect a President Romney?

Surely there are both bad and good Republican consultants, better and worse messages and tactics. What we need now is a means of determining one from the other. This is the real Democratic and Progressive advantage; the ability to distinguish between what works and what doesn’t using social science experiments.

Beginning with Progressive interest groups more than a decade ago, liberals allied with academic social scientists to study politics the way medical researchers study the effects of drugs. They embraced the core of the scientific method, deploying randomized, controlled experiments to rigorously test the effectiveness of their messages and tactics.

With these experiments, Democrats built – and continue to build – an objective base of knowledge on what works and what doesn’t. Today, the Left is permeated with a culture of testing. Expert opinions are taken as just that; opinions, hypotheses to be tested with experiments.

Why did the Obama campaign invest so heavily in grassroots field offices? Years of randomized-controlled experiments indicated that, for all their expense and difficulty, they gave the highest vote return. Why did they promote strange video pledges to support the president or send “voter report cards” informing Democrats of their voting record and how it compared with their neighbors? Experiments indicated that these are effective at getting out the vote.

None of these findings came from traditional campaign experience, or Big Data, or digital magic. They were the hard-won results of rigorous testing and thoughtful, creative social science.

So, where does the Right begin? First, we must recognize that we know far less about what works in politics, and what doesn’t, than we think we do.

Second, we must begin to design and execute experiments – hundreds of them – testing every major category of messaging and tactics.

Third, we must integrate these findings into comprehensive voter databases to ensure we have better data, not just bigger data. Only experimental data adds new information on how messages and tactics impact voters. It provides us insight from a counter-factual universe; what if every citizen was exposed to this bit of information or that ad? How would the vote shift? Who would turn out to vote or stay home? What we call “observational” data – the kind that currently drives micro-targeting – can never accomplish this, no matter how “Big” it gets.

At Evolving Strategies, we tested the impact of four Romney and three Obama ads on vote preferences and voter enthusiasm just before the first primary debate. One of these was a widely-aired ad produced by Americans For Prosperity, in which Obama voters express their disappointment with the president. The “Disappointed” ad consistently won favor in the traditional focus-group setting as a message that would shift swing voters and appeal to women.

Our randomized, controlled experiment, however, found the “Disappointed” ad didn’t move voters toward Romney. But it did increase enthusiasm among men who voted for McCain in 2008 and sapped the enthusiasm of male Obama ’08 voters.

Surprisingly, the “Disappointed” ad failed as the soft-edged appeal to swing voters for which it was intended, but seemed very effective as red meat for male voters in Romney’s base and for demobilizing Democratic men. Instead of airing the ad on networks that would reach a broad audience in the middle, they should have run it around the clock on ESPN.

What we know about political behavior is astonishingly little, even on the Left. What’s needed now on the Right is not an inquisition, but instead a scientific revolution in conservative politics. Success and accountability will come only with the sunlight of rigorous social science.

Adam B. Schaeffer is a co-founder and director of research at Evolving Strategies.