“I give them experiments and they respond with speeches.”

Louis Pasteur

Campaigns & Elections,: Movers & Shakers: Adam Schaeffer

image

Recent Media

» Evolving Strategies

Movers & Shakers: Adam Schaeffer

March 27, 2014

Schaeffer says experimentation can boost GOP campaigns, if only his side was more willing to…

Adam Schaeffer is the director of research and co-founder of the Republican firm Evolving Strategies.

C&E: Where are Republicans when it comes to testing and experimentation after the 2012 cycle? How do campaigners on your side tend to view it?

Schaeffer: We’ve advanced maybe five percent on this from 2012. Maybe it’s a little bit more, but there’s a long way to go. There has been a lot of talk about it, but it’s a radical change in perspective that took a long time to sink in, even on the left. Outside groups adopted it much more quickly than the campaign world—the unions, EMILY’s List or similar outside organizations. Campaigns are just their own thing. There’s more tradition, they’re faster paced and they disband almost immediately. So for all those reasons it hasn’t gone nearly as far as I thought it would by this point. Even on the data structure side, we need to get the basics nailed down. A lot of people are of chasing shiny new objects like consumer data, but the core of the voter file isn’t where it should be or could be.

C&E: What did you learn from the experiments your firm did over the last cycle?

Schaeffer: Well, this wasn’t a complete surprise, but we found differences between men and women in their reactions to get out the vote in a persuasion context. It was surprising to us just how extensive we found those differences. We also found negative impacts in terms of persuasion and get out the vote where you wouldn’t expect it at all based on the message. I think this is actually a huge blind spot in politics. I don’t hear the Democrats talking about this very much and I don’t really see much in the academic literature. People often think in terms of whether something is effective or ineffective. Well, it could be that something is not just ineffective, but it could actually have a huge negative impact on what you’re campaign is doing. I’m certain from what we found that there are plenty that campaigns and political organizations do that’s actually hurting their cause, but they don’t even realize it. You’re not just wasting money in this case, you’re spending money to hurt yourself. There’s an enormous amount of dead weight in campaigns. There are plenty of things that are not impacting people at all. If you can find those sweet spots in terms of get out the vote and in terms of your messaging, you can do a tremendous amount.

C&E: How do you pitch the experimentation piece for campaigns?

Schaeffer: It’s difficult. A lot of times there’s a notion they’re already doing this to some degree—polling, focus grouping, or even match control cases where they will do one thing in one precinct and something different in another. That’s certainly a lot better than not doing anything at all, but it’s dangerous because if you have small numbers and you’re matching things, bias creeps in. That can make all the difference. Experiments are deceptively simple. The concept is really easy to understand. But when you get down to what you are testing and then down to the analysis, it’s more complex. We always look at subgroup impacts. For example, do people who live with a Democrat in the household respond differently to something than people who live in an all-independent household? This is another area where we can advance. I know in academia there’s not much of this type of analysis. They tend to focus on the whole group average. Sometimes they only look at analyzing single-voter households.

C&E: And campaigns are often wary of anything that might seem too academic.

Schaeffer: When we’re pitching this to people, they’re either ready for it or they’re not. If they are, they immediately get it. It may be that something about the message or the strategy has been bugging them and yet they’re still spending a bunch of money on it. Or maybe it’s someone who lost a race and said, “Maybe we should have done something else.” Everyone has been in a room where two people have equally plausible arguments about why a campaign should absolutely do one thing over another. I think it’s more about finding the right fit. People have to understand this isn’t magic; it’s basic science. This is the same thing used for drug trials. Experimentation is what has built all of the technology we have currently, and we bring it to the practice of politics in a rigorous way, along with the behavioral aspect. We often find we’re not selling this as much as we are explaining it to people and showing campaigns how they can use it. The biggest hurdle can be selling people on a particular project. There are almost an infinite number of things to test, and there are different modes of contact and spending.

C&E: How much are you able to have experiments contained within a campaign so the results can actually inform the strategy or approach?

Schaeffer: GOTV experiments are obviously not really helpful within the context of a single campaign. We have had interest in conducting GOTV experiments in special elections where people mainly wanted to learn from it after the fact. If it’s an outside organization, they obviously have an interest beyond that single election, so that’s something of interest to them. For campaigns, it’s more the persuasion tests. And we can do this quickly. Sometimes it’s just difficult to figure out what a campaign wants to test. The connection between persuasion and get out the vote is a big one. In the short term, most of the value and impact any persuasion message is going to have is not necessarily taking voters from one side to the other, but mainly it’s convincing those marginal voters—the ones who may be leaning toward your opponent. I think the Democrats did that consciously with Romney voters.

The Obama folks knew that a lot of the people they were advertising to would never vote for their candidate, but not voting at all is a legitimate choice. People on both sides throw around the idea of vote suppression really easily. In my mind, vote suppression is lying to people or tricking them. Certainly there are folks who feel like a lot of the claims made about Romney were in that category, but a persuasive message that demobilizes the other side—that’s legitimate. There are a lot of people who don’t vote, and they don’t vote for legitimate reasons. So the connection between persuasion and get out the vote is big and it’s something that might be in the back of the minds of a lot of campaigns professionals, but I think it needs to be much more explicit.

C&E: Do you expect to see some sort of coordinated effort on the Republican side to invest in this sort of testing and experimentation in the near future?

Schaeffer: Maybe. To a certain extent, we would love to be that node in terms of disseminating things. I don’t know that we need something so central like the Analyst Institute, however. I also don’t know that it will develop that way on our side. It might just be a greater adoption of these techniques by companies and organizations over time. It’s just tough to tell, but I don’t think it’s necessary to really have a clearinghouse. The key is that we can disseminate information across channels. It’s the same thing with the whole data structure on the left—it got centralized quite quickly and it worked really well for them. There’s a lot of value in that, because you can get all your ducks in a row and perhaps leap ahead more quickly. I don’t know if that’s going to happen on the right. Maybe I’ll be surprised, but there are a lot of outside projects in terms of data. In a way it’s frustrating that we might be recreating the wheel, but it’s never really a bad thing to not have your eggs all in one basket.

C&E: Is the data piece of this a task that’s better suited for the party or for outside entities?

Schaeffer: I think there’s a real value in having tension between them. I wish we were further advanced than we are in both areas. The party organizations touch so many campaigns that it makes sense for them to do some basic stuff, especially with the voter file. Then again, information is getting cheaper. It’s now cheaper to obtain it, process it and utilize it. So I think we’re going to see something develop that’s an interplay between the parties and the outside groups. But we’re going to need multiple nodes for people to tap into.

C&E: What are you seeing on the digital side? Are campaigns actually changing their spending habits?

Schaeffer: Digital is just one aspect of what we need to do better. One of the problems is that there’s not enough clarity on what it works for and how to use digital dollars in the most efficient way. On the Democratic side I know they’ve done a lot of testing, but it changes so quickly and there are so many possibilities. You have pre-roll, Facebook, Google ads and all sorts of different platforms. It’s just more difficult to get on top of and systematically figure out where the best place to spend your money is. It’s also easy to ignore anything online that you don’t have an interest in, which is often the problem when we’re talking about political communication. How far can you go in forcing people to see a piece of information or reading something that they don’t want to? The trick is how to get them interested.

Make Pro-Abortion Extremists Play Defense

January 22, 2014

In 2012, Democrats ran a well-coordinated campaign to demonize and distort pro-life candidates as anti-woman misogynists hell-bent on taking away birth control. The Republican response to this line of attack consisted mostly of pivoting away to focus on “jobs” and the “economy.” With rare exceptions, instead of responding, GOP candidates were unwilling to answer the attacks head-on.

In order to win elections in the future, Republicans will have to change tactics and better respond to these scurrilous accusations.

They had a chance to change things in Virginia in 2013. Going into the Virginia governor’s race, pro-life advocates believed it would be a different ballgame with a strong pro-life leader in Ken Cuccinelli as the GOP nominee. During his career, Cuccinelli was known as a candidate unlikely to back down from a fight and unafraid to counterpunch.

On cue, and pulling from the 2012 playbook, Democrats pounded Cuccinelli with millions of dollars worth of “war on women” attack ads. The pounding was so severe that, just a couple of weeks before Election Day, the Cook Political Report found that McAuliffe’s campaign had spent more of its ad budget (26 percent) hitting the Republican on this topic than on any other issue. That is, McAuliffe and his allies ran more than 5,600 TV spots on abortion alone.

But instead of combating these so-called war on women charges, Republicans opted again to try to change the subject. McAuliffe’s echo chamber was complete when the press failed to cut through the paid media assault and relay accurate information about each candidate’s actual positions.

My organization, the Susan B. Anthony List, a national pro-life group, invested over $800,000 working to elect Cuccinelli. Knowing that the Republican candidate’s longstanding pro-life record would draw heavy fire, we sought to go on offense and directly counter the war on women charges. To start, in March of last year we commissioned a comprehensive poll with the goal of determining which messages could effectively drive votes away from McAuliffe.

Our $60,000 survey, conducted by Public Opinion Strategies, tested 15 different negative messages about McAuliffe. We tested topics such as McAuliffe’s support of Obamacare, his union ties, his GreenTech scandal, his shady Global Crossing business dealings, and his support for gay marriage. We also tested how voters reacted to learning of McAuliffe’s extreme stances on abortion.

Our abortion messages ranked high on the list, often within the top two most effective messages among key voter demographics, in turning voters away from McAuliffe. Among our findings:

· Northern Virginia women: 51 percent of Northern Virginia women said they were much less likely to support McAuliffe after hearing he supported sex-selection abortion. This message ranked second most effective out of the 15 messages tested among these women. The fifth most effective message with this group was McAuliffe’s support of taxpayer-funded abortion, with 41 percent of Northern Virginia women saying they were less likely to support him.

· Independents: 55 percent of independents said they were less likely to support McAuliffe after learning he supported sex-selection abortion. This ranked second out of the 15 messages. The fourth most effective message among this group was McAuliffe’s support of taxpayer-funded abortion, with 44 percent saying they were less likely to support him.

· Undecided voters: McAuliffe’s support of sex-selection abortion was the most effective message with this group, with 56 percent saying they were less likely to support McAuliffe. The issue of taxpayer-funded abortion ranked third with this group, with 49 percent saying they were much less likely to support McAuliffe.

· Republicans: Among base Republicans, the issue of taxpayer funded abortion ranked second in turning them away from McAuliffe, with 86 percent saying they were much less likely to support him after hearing this message. Sex selection was a close third. Among soft Republicans, sex-selection abortion ranked fifth, with 67 percent less likely to support McAuliffe.

Among voters who initially told the pollster that they were supporting McAuliffe but backed down from their support upon hearing these messages, 66 percent voluntarily cited disagreement with McAuliffe’s abortion stance as a reason. This ranked highest of any issue volunteered by respondents.

These results clearly showed that by going on offense on abortion and exposing his opponent as extreme, Cuccinelli could have not merely energized the base, but also turned suburban women, independents, and undecided voters against the Democratic candidate.

After commissioning this extensive survey, we dug deeper by conducting two focus groups in Loudoun County, a key swing area in northern Virginia. Our first focus group was with Republican-leaning independent women, and our second was with Democratic-leaning independent women.

With both groups, we outlined each candidate’s actual position on abortion, without bias, and asked each to identify who they believed was the abortion extremist. Republican-leaning women said it was McAuliffe; the Democratic-leaning women labeled both candidates extreme. The focus group exercise reinforced the findings of our poll and proved the abortion issue could have been a net-positive for Cuccinelli.

Following our polling and focus groups, we aggressively got the results into the hands of the Cuccinelli campaign, the Republican Governors Association, the Virginia Republican party, and outside groups interested in the race. What feedback did we receive on this solid data? None.

The SBA List charged forward, spending the bulk of our resources making direct contact with voters – reaching over a million by phone and nearly 70,000 at their front doors. Our message was to expose the extremes of McAuliffe’s support for abortion on demand, up until the moment of birth, including sex selection and taxpayer-funded abortions.

Weeks after Cuccinelli’s loss, a column published in Campaigns and Elections magazine confirmed the essentials of the SBA List’s strategy. The outcome of the election could have been different if those calling the shots in the GOP had worked with us to play offense on abortion and spent more of their own resources spreading the same message.

Experimental research by Evolving Strategies and the Middle Resolution PAC found that what most moved voters away from Terry McAuliffe was detailing his extreme pro-abortion position:

“A single phone message emphasizing McAuliffe’s support for unrestricted, late-term, and taxpayer-funded abortions shifted support a net 13 to 15 points away from McAuliffe and toward Cuccinelli… A topic declared radioactive by nearly everyone, locked away in secure storage behind a blazing Hazmat warning by the Cuccinelli campaign, appears to have been a powerful weapon for the Republican ticket that could have substantially closed the gap, and possibly even won Cuccinelli the election.”

The authors emphasized the urgent need for the Republican party to aggressively cull and refine dynamic voter data through experiments. Data-driven campaigns, they argue, built out through a system of creative targeting and messaging tests, must replace the GOP’s preferred strategy of targeted silence the last several cycles. Blindly following the gut instinct of the consultant class to ignore social issues is failing miserably. More resources funneled towards an aggressive pro-life message could have made the race winnable.

The evidence suggests the other side had similar data that it didn’t ignore. Schaeffer and Smith pointed out that the narrowness of the race was a surprise for nearly everyone, save perhaps McAuliffe’s own data team. It makes sense now why, just as Cuccinelli was making a comeback in the polls prior to Election Day, the Democratic party of Virginia launched a deceitful robocall campaign. In an effort to suppress the GOP base vote, the call went out to conservative, pro-life voters alleging – of all things – that Cuccinelli was not truly pro-life.

The data-backed evidence is clear: Abortion can be a winning issue for pro-life candidates. Going into 2014, will the GOP’s consulting class finally take notice after two losing cycles, dominated by the war on women narrative?

As long as abortion continues in this country, so too will the restlessness in the hearts and minds of Americans. The issue will not go away until the injustice itself is ended. The good news is that a huge opportunity awaits the candidates, consultants, and party prepared to fight.

Following the grisly details of barbaric fetal homicide in Kermit Gosnell’s “house of horrors,” which received national attention, lawmakers took action across the country to end brutal late-term abortions last year. The U.S. House of Representatives passed the Pain-Capable Unborn Child Protection Act, which seeks to protect unborn babies capable of feeling pain beginning at 20 weeks, that is, more than halfway through pregnancy. This legislation has been passed by over a dozen states, and has been introduced in the U.S. Senate by Sen. Lindsey Graham, with 41 co-sponsors.

Sen. Graham’s goal is simple: Force a vote prior to the 2014 mid-terms and put every Senator on record. Sen. Kay Hagan of North Carolina and Sen. Mark Begich of Alaska, both vulnerable Democrats up for reelection, have already said they would vote against it. Other vulnerable senators, such as Mary Landrieu of Louisiana and Mark Pryor of Arkansas, have not taken a position.

Five national polls conducted in 2013 show overwhelming support for this legislation, especially among women, independents, young people, and Latinos.

This legislation will give the Republican party a new opportunity to identify senators who vote against this legislation as abortion extremists. By going on offense, Republicans can flip the war on women narrative on its head and put the other side on defense. Or they can continue, ironically, to curl up in the fetal position, let their own personal bias against social issues trump the data, and continue to lose. Time will tell which path they choose.

Marjorie Dannenfelser is president of the Susan B. Anthony List, a nationwide network of more than 365,000 Americans dedicated to pursuing policies and electing candidates to reduce and ultimately end abortion.

Vaccines vs. Leeches

December 9, 2013

Why the left is strong and the right is sickly when it comes to campaigns

Virginia’s 2013 gubernatorial election was much closer than anyone, with the possible exception of Terry McAuliffe’s data team, expected. Rather than the 7-point drubbing the poll averages suggested, it came down to about 55,000 votes out of more than 2 million cast, a 2.5 percent margin.

This relatively close call has given rise to all manner of what if questions: What if Ken Cuccinelli had more money? What if Governor McDonnell hadn’t been embroiled in scandal? What if the shutdown didn’t happen?

But none of these questions get to the core problem with the Cuccinelli campaign. Indeed, the core problem with almost every Republican campaign to date: the absence of randomized-controlled experiments.

True experiments produce dynamic data, which tell us how to move an electorate and what to do to win an election. Experiments have been at the core of the progressive revolution in modeling, targeting, persuasion and turnout over the past decade.

In fact, the McAuliffe campaign plan—the messaging, targeting, and tactics—was built around extensive experimental findings. And yet it remains exceedingly rare to find anyone on the right who is aware of applied political experiments, let alone someone who understands and utilizes them.

In Virginia, many have argued the Obamacare debacle helped close the gap in the final weeks of the gubernatorial election, and that with more time and money to get Cuccinelli’s message out on that front, he might have won. But this is pure speculation.

Correlation doesn’t prove causation, and what we have here is a correlation between the media’s focus on problems with Obamcare and an unexpectedly close result in the gubernatorial election. Absent experimental data, we can’t say whether these events impacted the race or whether messaging tied to it would have shifted more votes toward Cuccinelli.

Evolving Strategies and the Middle Resolution PAC conducted experimental research that suggests an aggressive attack on McAuliffe for supporting ObamaCare was ineffective at best and counter-productive at worst. An attack on McAuliffe’s business record possibly helped, but was anemic.

What moved the voters most was an attack on McAuliffe’s positions on abortion; a single phone message emphasizing McAuliffe’s support for unrestricted, late-term, and taxpayer-funded abortions shifted support a net 13 to 15 points away from McAuliffe and toward Cuccinelli. The cost per vote here was a remarkably cheap $0.50 per additional vote, and even less expensive still when targeting the most persuadable segment of the electorate.

A topic declared radioactive by nearly everyone, locked away in secure storage behind a blazing Hazmat warning by the Cuccinelli campaign, appears to have been a powerful weapon for the Republican ticket that could have substantially closed the gap, and possibly even won Cuccinelli the election.

Here is the fundamental lesson: We do not know what works until we test it, repeatedly, using experiments. Randomized-controlled experiments allow us to block out all the other noise and pinpoint precisely how a message or tactic changes voter behavior. This is key: rigorous experiments randomly assign voters to receive a treatment, or to a control group that receives nothing, or a placebo.

Quasi-experimental approaches, such as match-control groups where a set of voters or precincts is matched on observable characteristics, fall far short of the certainty and precision we need. But a true experiment, with large enough numbers, allows us to statistically compare the outcomes in the treatment to those in the control group, and any difference between the two can be confidently attributed to the treatment alone.

Democrats and the broader collection of progressive organizations attack issues and elections as problems to understand and then solve. They conduct rigorous, aggressive research on the composition and disposition of the electorate well in advance of an election. The right still largely relies on gut instinct and guru-ism, where the argument goes to the most forceful personality and best storyteller. This is changing, but too slowly.

Earlier this year, a national grassroots organization, Evolving Strategies, and Middle Resolution worked together to conduct experiments in three Republican primary elections for the Virginia House of Delegates. Each race featured 20-year-plus incumbents with deep pockets and the full backing of the governor given their support of his controversial transportation bill and tax increase.

In each race, we backed a challenger with strong grassroots support, but fewer resources and little or no political experience. We conducted experiments testing the impact of yard signs, GOTV text messages, and GOTV survey scripts guided by voter psychology experiments and innovative hypotheses. Two out of the three incumbents we targeted lost their seats.

This year, Evolving Strategies has tested mail, texts, GOTV surveys, radio ads, and robocalls for persuasion and GOTV in New Jersey, Massachusetts, Colorado, and Minnesota. Each experiment adds to our base of knowledge and provides a basis for testing even more promising possibilities for turning out or persuading voters more efficiently.

Our winning survey script, for instance, boosted turnout for our candidates by 12 points over the control group, and we tested additional variations during the Virginia gubernatorial race. Our cold GOTV text messaging impacts range from -15 to +15 points depending on the message and the targeted demographic. We have found massive differences in the impact that a message has on particular kinds of voters.

In the absence of experiments, we can’t know whether we’re helping or hurting our cause. With experimental testing, however, we discover not only what’s effective, but what’s most efficient and with what kinds of voters. As we learn and test more, we reap returns from a ratchet effect, discarding what fails and improving on what succeeds.

To loosely paraphrase pioneering scientist Louis Pasteur, the left relies on experiments to determine strategy and tactics, and the right relies on speeches. The right largely remains mired in a pre-scientific age of just-so stories and persuasive conjecture, using theories that are the electoral equivalent of the four humors and tactics the equivalent of leeches. Things are beginning to change on the right, but not nearly as quickly or broadly as is necessary to start winning in races like Virginia in 2013.

That’s not to say we won’t win a lot of races next year. Campaign strategy and tactics can’t make an unpopular president or healthcare debacle magically disappear. After all, some patients sick with smallpox survived in spite of being bled with leeches, but far fewer than those in later years who would be vaccinated against the disease. We will lose seats that are winnable, and that might mean the difference in control of the Senate.

There’s no shortage of changes that Republicans and the right might make for the better, but there are none more important for winning minds and elections than a full embrace of true experiments and rigorous research.

Adam B. Schaeffer, Ph.D., is a co-founder and director of research at Evolving Strategies, LLC. Nancy Smith is Grassroots Coordinator for Middle Resolution PAC.

Republicans Need to See Big Data as Big Business

July 16, 2013

Michael Simon, 33 in September, runs one of the near-dozen new firms that deal in the Big Data that helped power President Obama’s re-election. HaystaqDNA is now busy helping corporate clients identify potential new customers. Asked what he hopes will become of what he and fellow Obama alums built, Simon says, “lasting dominance,” so that “when the time comes to go back to Iowa, we will still be the leading edge of what’s possible. Republicans never catch up.”

Adam Schaeffer, 36, feels “grossly, vastly outnumbered.” His firm, Evolving Strategies, conducts field and online experiments to improve persuasion, mobilization and voter targeting for center-right clients. Asked what he hopes his side will achieve with analytics in the next few years, he says, “I’d be very happy getting halfway to where the left is by next year on all these fronts.”

Schaeffer believes this is doable. Off-year and midterm elections offer “cleaner” environments in which to model without the saturation of a presidential race. Less information and contact from the media and campaigns means any effects of experiments would be easier to spot. He’s just not sure the galvanizing will exists to knit scattershot efforts into something more lasting.

More senior Republican targeters agree: It doesn’t. While Democrats have a flourishing industry of privatized data banks, serial experiments and competing consultancies, the party that espouses free markets and entrepreneurship is on Stalin’s Five-Year Plan when it comes to embracing Big Data. Democrats saw the viable business model of moving data outside the party but selling it to all friendly comers. Republicans basically moved their data out of the party in name—i.e., the Data Trust—and tax status only as a way to manage the cost of maintaining it.

As a result, analytics remain bottled within the national Republican party structure. One respected party entrepreneur half-jokes that his fellow Republicans should remember the history lessons of Teddy Roosevelt’s “trust busting” and “free the GOP marketplace to create the best data and tools for every Republican candidate.”

The main problem, he says, is that GOP state chairs historically have used access to the voter files not to win other offices but to keep their own, doling it out to local party and elected officials. Far from automatically winding up in the hands of their campaigns and consultants, the files often don’t. Candidates don’t benefit from the information. Operatives don’t grow comfortable working with the data. And the great potential of analytics to help campaigns figure “how you best spend your resources,” as Simon describes it, is squandered and forfeited to the left.

The problem has worsened as the Republican National Committee has refined its focus to the presidency, the GOP entrepreneur says. The voter files are more up to date for battleground states (of which there weren’t many in 2012) than for states where demand otherwise would be high.

Also, as a rule, this entrepreneur says, Republican candidates who do gain access only do so once they’ve become nominees. At that point, if they’re granted access, it’s free. The prospect of eventually getting free data keeps campaigns from buying it from private sources, stunting the potential growth of consultancies. It also keeps candidates hooked on the free data regardless of its quality.

Republicans’ initial push on analytics was forceful. Determined to follow up on their 2000 victory with a more emphatic 2004 win, the RNC pumped money into companies such as TargetPoint Consulting and National Media Research, Planning and Placement.

(Check out our Big Data At-A-Glance for the most up-to-date roster of who’s who and where in campaign analytics.)

But after 2004, they took their foot off the gas while Democrats gunned it. Campaign finance developments were forcing progressive interest groups to hunt for more small-dollar donors. That spawned Catalist and the Analyst Institute. Unions in particular, facing shrinking membership, drove the party’s embrace of modeling and experimentation.

In Chicago, the embrace of technology and analytics was prompted by the needs of an upstart presidential contender who faced off against Hillary Clinton and the Democratic consulting establishment. “We had no gurus,” Simon says. More than just new approaches, Obama’s historic candidacy brought out atypical talent—the kind of talent Republicans are more painstakingly recruiting. (Ask any Obama analyst, what did you do before 2012? The answer is as likely to be “particle physicist” as anything else.)

Credit the RNC for seeming to understand that they’re leading by default. Their laudably realistic 2012 post-mortem report calls for more election-season experimentation, including in urban markets during the 2013 big-city mayoral races, and for new approaches to recruiting talent. The report’s weakness is that it suggests, understandably, these functions remain under party control.

The silver lining for Republicans? Schaeffer points out, “There’s a lot that we do know that’s in the public sphere” thanks to published academic research, often in partnership with interest groups on the left, and best-practice guidance from progressive efforts like the New Organizing Institute (NOI).

 

It’s also thanks to Democratic analysts who, in the interest of ginning up business or just getting credit, have told reporters some ingredients of the secret sauce. Compare that to the Republican strategist who I heard might be doing some promising experiments but who, when contacted, replied, “Sorry but I am not talking about that publicly and nor should be the person you heard it from, ha.”

(Check out The Cook Political Report‘s Big Data At-A-Glance for our list of the major players and new faces in the world of political Big Data.)

The time is ripe for making up ground, and Simon recognizes that if the left doesn’t keep innovating, “Republicans could quickly catch up.” Not only do 2013 and 2014 offer fallow off-year fields in which to experiment, but 2016 dangles the kind of opportunity the Obama team faced in 2008: an open presidential primary in which risk-taking and innovation could pay off for some candidate in spades. Donors are more interested in backing research during big election years, Schaeffer notes.

A timely investment in analytics also could help Republicans with some of their demographic challenges. As Simon points out, “analytics enable us to be less ham-handed and cheesy, to speak to people as individuals and not so much as groups”—to treat people on all the various factors that make up their respective identities rather than focus on just one.

The seasoned entrepreneur suggests that to the extent the left’s analytics renaissance was Obama-centric, it’s not clear that the inspiration or the data will transfer to future presidential nominees. He favors—as the RNC’s new Facebook alum chief technology officer apparently also does—making the voter files directly available to campaigns and consultancies to finally help nurture an industry that will exist independently of any candidate.

Schaeffer believes the burden of proof is on firms like his own. “Our job this year is to really demonstrate what can be done and what can be learned.” But is the burden really on data entrepreneurs like him? So long as data is treated as a means of keeping parochial influence, Republicans may never launch the kind of industry needed to help them remain nationally influential.

Obama Does It Better

October 29, 2012

When it comes to targeting and persuading voters, the Democrats have a bigger advantage over the GOP than either party has ever had in the modern campaign era.

A few weeks ago, two Washington Post journalists who barely three months before Republicans lost control in Congress in 2006 released a book calling the party “the New York Yankees of American politics—the team that, at the start of every season, has the tools in place to win it all,” reported that the right is back. Their story, headlined “Conservative groups reaching new levels of sophistication in mobilizing voters,” presented a roster of outfits whose efforts could prove a counterweight to Barack Obama’s fearsome ground program.

Two days later, the New York Times described one of these groups in a story of its own. In 2009, scandal-tarnished former Christian Coalition impresario Ralph Reed founded the Faith and Freedom Coalition and, according to the Times, “now plans to unleash a sophisticated, microtargeted get-out-the-evangelical-vote operation he believes could nudge open a margin of victory.” The Times story explained in unusual detail the method behind Reed’s data operation:

To identify religious voters most likely to vote Republican, the group used 171 data points. It acquired megachurch membership lists. It mined public records for holders of hunting or boating licenses, and warranty surveys for people who answered yes to the question “Do you read the Bible?” … It drilled down further, looking for married voters with children, preferably owners of homes worth more than $100,000. Finally, names that overlapped at least a dozen or so data points were overlaid with voting records to yield a database with the addresses and, in many cases, e-mail addresses and cellphone numbers of the more than 17 million faith-centric registered voters—not just evangelical Protestants but also Mass-attending Catholics.

Those who have actually worked with voter data were a bit less awed by this description of Reed’s process. One Republican consultant describes it as “backward microtargeting.” Acquiring membership lists from allies is a decades-old practice in coalition politics, and the central tactic—sending voter guides to people on church rolls—last seemed cutting-edge when Newt Gingrich’s career was first on the ascendancy.

“There is nothing new in that article,” says one veteran of the Bush campaigns who spoke anonymously to candidly critique a fellow Republican’s program. “It was pretty much what we did in 2000.”

Indeed, the Reed approach seems oblivious to the most important innovations that have taken place in the years since. Microtargeters often describe their project as “look-alike modeling,” because the goal of using statistical algorithms is to discern patterns in an existing sample (like people on a church list) that can then be used to find people who resemble them in other populations, about which there is less information available. There is significantly less value in acquiring data that confirms that your targets look the way you thought they would.

The consequence of such primitive targeting was felt recently at one mailbox in the Richmond suburbs. The letter was addressed to a woman who attends Mass and subscribes to a Catholic Charities newsletter, is married with children, and lives in a home worth more than $100,000. She may have racked up a lot of points in Reed’s categories, but there’s one other publicly available fact about her—she regularly votes in Democratic primaries for federal and state office—that an algorithm would likely have treated as more predictive of her political attitudes than her income or church affiliation. Reed’s get-out-the-vote mail had targeted a phone-banking Obama supporter.

All targeting carries the risk of missing the mark, and there are regularly voters whose actual attitudes defy the predictions of statistical models. But regular misfires by Republicans—which at best only waste resources and at worst mobilize Democrats who might not have voted otherwise, or provoke a backlash among those still persuadable—illustrate a gap between how the right and left practice politics in the 21st century. Contrary to the wishful intimations of the Post and Times stories, while the groups on the right could conceivably catch up with Obama and his allies in the scope and funding of their ground-level activities, in terms of sophistication they lag too far behind to catch up in 2012.

In fact, when it comes to the use of voter data and analytics, the two sides appear to be as unmatched as they have ever been on a specific electioneering tactic in the modern campaign era. No party ever has ever had such a durable structural advantage over the other on polling, making television ads, or fundraising, for example. And the reason may be that the most important developments in how to analyze voter behavior has not emerged from within the political profession.

“The left has significantly broadened its perspective on political behavior,” says Adam Schaeffer, who earned graduate degrees in both evolutionary psychology and political behavior before launching a Republican opinion-research firm, Evolving Strategies. “I’m jealous of them.”

Photo by David Greedy/Getty Images

Schaeffer attributes the imbalance to the mutual discomfort between academia and conservative political professionals, which has limited Republicans’ ability to modernize campaign methods. The biggest technical and conceptual developments these days are coming from the social sciences, whose more practically-minded scholars regularly collaborate with candidates and interest groups on the left. As a result, the electioneering right is suffering from what amounts to a lost generation; they have simply failed to keep up with advances in voter targeting and communications since Bush’s re-election. The left, meanwhile, has arrived at crucial insights that have upended the conventional wisdom about how you convert citizens to your cause. Right now, only one team is on the field with the tools to most effectively find potential supporters and win their votes.

***

The first dramatic expansion of the campaign brain in the 21st century came from the world of commercial marketing. Private-sector data warehouses, created initially to generate credit ratings and later used by direct-mail marketers, had collected far more information on voters than had ever been available to campaigns through traditional political sources. Improvement in database architecture and computing power made it possible to run statistical models that could churn through tens of millions of these consumer records at once. While operatives on both sides tapped into this capacity, it was Republicans—thanks in part to close ties between some of the party’s public-opinion researchers and the private-sector firms that agglomerated consumer data—who fully exploited its potential first.

Following Bush’s re-election in 2004, Democrats worked assiduously to catch up with what they considered the Republicans’ structural data advantage, developing their own relationships with commercial data vendors and refining their algorithms. Today, the most advanced political campaigns have in certain respects surpassed consumer marketers in their ability to predict individual preferences, and you’re as likely to see a Fortune 500 company trying to uncover the secrets of the Obama data operation as the other way around.

Yet the campaign brain has continued to expand. The most important methodological and conceptual breakthroughs in recent years have originated in the academy, specifically through insights from behavioral psychology and the use of field experiments. Since 2004, myriad advocacy groups and consulting firms on the left have joined forces and launched a series of nominally for-profit private research institutions devoted to campaign tactics. The most impressive among them, the Analyst Institute, was created to link the growing supply of academics interested in running randomized-control trials to measure the efficacy of political communication with the demand of left-wing institutions eager for empirical methods to test their programs. These partnerships have birthed a generation of political professionals—many baptized in the unprecedented pools of data collected by Obama’s 2008 effort—at ease with both campaign fieldwork and the techniques of the social-science academy.

This summer, a top Republican analyst stumbled upon a job notice posted by the left-wing League of Conservation Voters. The position was Targeting and Data Director. The analyst looked admiringly at the description of the job, especially its duties to “explore and devise opportunities to test and measure the impact of all of our programs, including working closely with entities such as the Analyst Institute.” He marveled at what that language revealed about the sophistication of his rivals’ intellectual enterprise. “One thing the left—Catalist, Analyst Institute, New Organizing Institute—has done very well is training and seeding of this sort of stuff, this sort of philosophy,” said the analyst, who asked not to be identified because of election-season attachments but has worked closely with the Republican National Committee and presidential campaigns.

Dozens of such postings exist in what some call the “progressive data community.” I asked the Republican analyst what analogous jobs existed among the institutions of the right. How many of the League of Conservation Voters’ ideological foes—like the Chamber of Commerce, or their frequent allies at the National Rifle Association or the Faith and Freedom Coalition—have data managers and targeting directors with similar mandates to test and measure?

“I honestly don’t know,” the analyst replied. “If I had to guess? Zero.”

The Analyst Institute’s centrality in the left’s research culture has enshrined the use of randomized field experiments as the best tool for measuring what actually moves voters. And the biggest conceptual contribution this body of experimental work has made is to cleanly separate what a voter does in election season into two discrete phases: choosing among candidates and deciding whether to vote. Experiments have shown that giving voters more information about candidates or issues or the stakes of the election does little to adjust their likelihood of casting a ballot. To budge a nonvoter out of complacency, campaigns have learned, they have to use psychological techniques focused on getting someone to do something he or she is not used to doing. There’s one set of tools for changing opinions, and another for modifying behavior.

This basic paradigmatic distinction appears lost on many of those who direct campaign activity on the right. In an interview, Ralph Reed said that the Faith and Freedom Coalition’s microtargeting project had identified 10 million citizens in 18 states comprising a “turnout universe” that would receive at least six get-out-the-vote contacts, starting with a voter guide outlining differences between Obama and Romney on 10 issues, including taxes and abortion. “We’re looking for anyone who is registered to vote and who would benefit from this information,” he explained. I asked why, if the goal was to mobilize infrequent voters who had already been profiled as likely to be socially conservative, was he sending them information designed to persuade them that Romney was better on the issues they cared about. “I don’t know if it could be called persuasion,” he replied. “We think we need to educate them on where the candidates stand.” If Reed had been aiming to play dumb in the interview to obscure his group’s tactics, he succeeded.

***

In August, a Virginia playwright and newspaper editor named Dwayne Yancey was surprised to see a series of glossy direct-mail pieces from the Romney campaign arrive at his home outside Roanoke. The first two brochures had to do with coal mining, which struck Yancey as irrelevant to him or his family: They live four hours from the nearest mine, and coal production carries little of the romantic imagery for Yancey that have led Republicans to believe it was a potent issue in West Virginia and Kentucky. “At first I thought it was simply urban ignorance of rural Virginia,” says Yancey, who wrote about the mailers on the website of the Roanoke Times, where he works. Then more mail came to the Yancey household from Romney’s campaign, on more plausible subjects: about the deficit, about Medicare and Social Security, and one item that attacked Obama for being “All Welfare. No Work.” The thing that puzzled Yancey most about all of the Romney mail was the person in his household had been selected to receive it. Every piece, from coal to welfare, had been addressed to his 23-year-old daughter.

His daughter, whom I’ll call Sarah, settled on her choice in the presidential race long ago, but the campaigns had no way of knowing that. Sarah is private about her political views; she does not respond to phone calls from campaigns and does not believe she has given the Romney or Obama camp an indication of her preference through any other channel, like signing up on a website. She remains something of a cipher in the piles of data through which campaigns sift in the hunt for clues about how they ought to engage her. Virginia does not allow citizens to register with a political party, and although Sarah had voted by mail in every election since turning 18 she had voted in only one primary, for the Democrats’ 2008 nomination. She has refrained from partisan activity like donating to a candidate or joining an ideologically-minded membership group. And given a limited buying history and the fact that she has never had a home in her name there was scant information in the consumer databases that often help round out a voter portrait. (Yancey provided me with his daughter’s name so I could see what information was available about her in political and commercial databases, and she discussed her views so long as I agreed not to publish any of her identifying characteristics.)

The available information about Sarah would seem, at first glance, to produce conflicting indicators. Sarah is a young woman and has voted in a Democratic primary, which would probably point her toward Obama. But she lives in a precinct that votes overwhelmingly Republican, which would nudge things back toward Romney. The algorithms that automate these assessments based on available data are similarly indecisive. One Democratic targeting firm’s statistical model predicts a 59 percent likelihood Sarah would self-identify as a Democrat.

Yet only one of the two presidential candidates looked over the summer at this profile of a centrist, albeit one who leaned slightly left, and saw a voter whose mind was up for grabs. The Romney campaign had concluded that Sarah was the type of voter it could persuade—either because she was actually undecided among the candidates, she was a soft Obama backer who could be convinced to defect, or a soft Romney backer whose support needed to be shored up. Yet the Obama campaign didn’t launch any parallel efforts to persuade this supposed middle-of-the-roader. Democratic targeters may have looked at her and concluded that her vote was not in question—or that the campaign had a better way of reaching her to make its case than issue-based direct mail. Those calculations, and the years of experimental findings informing them, may reflect better than anything the massive gap between how Democrats and Republicans understand the challenge of finding voters to convert to their sides.

***

The differences in the two campaigns’ approaches to Sarah Yancey may owe more to Aaron Strauss than anyone else. Strauss is precisely the type of person who does not go to work in Republican campaigns. In 2004, the recent college graduate had used his skill with computers to manage Howard Dean’s New Hampshire voter information, earning the nickname Data from the campaign’s field organizers. Afterward Strauss went to work for Mark Mellman, who served as the lead pollster for John Kerry’s presidential candidacy. Then Strauss returned to school, choosing to study at Princeton with Kosuke Imai, who seeks methodological solutions to the ongoing problems political scientists face when trying to isolate cause and effect amid the fog of elections. (Representative paper title: “Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis.”)

For his dissertation, Strauss set out to run a randomized-control experiment measuring the effect of get-out-the-vote reminders sent by text message before the 2006 midterm elections. Most of the early get-out-the-vote experiments conducted by political scientists measured the average impact of a given approach across the whole population that received it. But Strauss knew from his work with Dean and Kerry that computerized voter lists had made it possible to segment voters based on their unique attributes. With co-author Allison Dale, Strauss layered that individual data into the experiment’s design to refine its insights into cause and effect. On average, their text-message reminders increased turnout among recipients by three points. But using voter data, they confirmed empirically what may have previously been mere instinct: The biggest impact was felt among digital natives, voters between ages 20 and 24.

Photo by Saul Loeb/AFP/Getty Images.

Strauss earned his doctorate and returned to politics, where experimenters at liberal institutions had begun toying with similar statistical methods. Unlike university researchers, they weren’t limited to nonpartisan mobilization exercises; they could also try to change a voter’s mind about who to support. And instead of merely looking at the demographic variables available in voter-registration records, partisan experimenters can also overlay the results of microtargeting models that sift through hundreds of data points to generate “support scores”—a percentage probability that an individual would back the Democratic candidate.

The people who first developed the microtargeting models used in persuasion had assumed, like the rest of us, that voters in the center are the most up for grabs. But in 2006, EMILY’s List ran a series of persuasion experiments that raised doubts about this assumption. The Democratic women’s group sent out mailers on behalf of female gubernatorial candidates in Michigan and Washington, then polled across the entire universe of recipients to gauge the impact of the messages.

The voters who’d been assessed as sitting closest to the middle of the road barely budged. In fact, there was significantly more movement among those who were projected to be leaning toward the Republican candidate than among those whose mid-range scores situated them evenly between the two poles. “Campaigns love to find out what segments of the population are their targets,” Strauss told me last summer in an interview for my book The Victory Lab. But that alone, he went on, was insufficient. “Targeting is all about finding people whose behavior will change and changing that behavior.” And it turned out that the people who’d scored close to 50 on the zero-to-100 spectrum of support weren’t the people whose behavior was most likely to change. Whatever those support scores were measuring, it wasn’t exactly susceptibility to persuasion.

Again working for Mellman, Strauss designed an experiment to test voters’ responsiveness to messages used by Harry Reid’s campaign as it prepared for a tough 2010 re-election to the Senate. Afterward, by identifying the attributes of voters who changed their opinions of Reid when presented with his primary campaign themes in so-called “trial heats,” Strauss was able to develop a “persuasion score” for all Nevadans, estimating an individual’s probability of being moved by that message. “Everybody has the probability of being the type of person that could be persuaded by this messaging,” Strauss explained. “So you want to give everyone a score, a probability of being persuaded, and also a probability of being unmoved and additionally a probability of being anti-persuaded—that they will counter-argue the message and actually move away from you.”

The Reid experiment, and a simultaneous test during Kentucky Congressman Ben Chandler’s re-election, reaffirmed the initial EMILY’s List finding: A campaign couldn’t just scoop up people who appeared to reside in the political center and assume they were all persuadable, an assumption now discredited in Analyst Institute circles as the “middle-partisan fallacy.” “It’s not always guaranteed that the people in the middle of a support score are the most persuadable,” Strauss said. “Some messages work really well to prevent defections on your side, so they would work best on people with high support scores. Some messages work best at promoting defection, so they work best on people with low support scores. And some messages do honestly work best in the middle, but you don’t know what kind of message you have ahead of time.”

Strauss began to look at why voters might be showing up with those middle scores but not moving. He thought of them as existing in three different categories. Some voters had simply remained relatively anonymous, with little data about them on file to push them toward one candidate or another, while others existed in demographic categories that did not contribute meaningfully to predictions about their politics. (One example Strauss uses is voters in their 30s, “an age range that neither leans Democrat nor Republican.”) In both of these cases, the middle-range scores gave a misleading indication that a voter was persuadable. “I have to always remind people that 50 means we don’t know, not that someone is evenly divided,” says one Democratic state-party data manager.

Democratic party volunteers Chris Lettero, left, and Matt Lattanzi knock on apartment doors while canvasing for votes Sunday in Youngstown, Ohio. The volunteers canvased door to door, a day before President Obama’s scheduled campaign rally in Youngstown. Political analysts have predicted Ohio could potentially decide the upcoming Presidential election.There was a third case, though, in which there could be lots of data available about an individual voter that effectively cancels itself out. These situation resembled the predicament that political scientists have long defined as cross-pressure, where a voter’s choice is complicated by conflicting aspects of his or her identity—the African-American who’s also a Mormon, to take one example. In these cases, a mid-range score seems to quantify precisely the type of ambivalence that makes for a good persuasion target. “We certainly want to talk to voters who are cross-pressured,” Strauss has written.

The hunt for persuadable voters has taken Strauss to the Democratic Congressional Campaign Committee, whose staff he joined earlier this year after leaving Mellman’s firm. In June, the DCCC ran what Strauss called a “persuasion-microtargeting experiment,” to test Democratic messages on voters in the field. Experiments found pockets of voters who moved in their direction in response to particular appeals: After hearing the party’s message on Medicare, men over the age of 65 increased their support for a generic Democratic congressional candidate three points more than the broader population. The DCCC could build a profile of voters whose opinions it could change, even if the data about them didn’t portray them as perfect centrists.

Only through such experiments that try to push voters and wait to see which ones moved can targeters know which voters were actually persuadable, and to what messages. At the moment it appears that only one side has embraced experiments for campaign research, and in his job as the DCCC’s data and targeting director, Strauss is in a position to institutionalize within his party’s campaign culture the skepticism he has developed about traditional targeting logic. “I actually think this is a competitive advantage we have right now over the Republicans,” Strauss, who would not comment for this piece, said last summer. “That competitive advantage might not last, but I think we have a competitive advantage over them.”

***

Republican operatives around the country have noted with a mixture of curiosity and anxiety that nearby mailboxes are less crowded with mailers making the case for Obama than they were four years ago. This may be the result of a strategic imperative: In many states, Obama has a clearer path to victory than Romney solely by mobilizing existing supporters than by finding new ones. But it could also reflect the fact Obama’s strategists do not think they have to rely, as have most campaigns over the last generation, solely on the mail for their targeted efforts to win over voters.

Earlier this year, Obama put his volunteers’ ability to do that to the test. The campaign administered an experiment in several states in which phone-bank volunteers were given a script with a few talking points and broad instructions to open up a conversation with a potential voter. Before and after these interactions, a professional call center surveyed the targeted voters to identify which candidate they supported, and campaign analysts set to work developing a statistical portrait of those who moved in Obama’s direction after talking with a volunteer.

 The result of that analysis is the campaign’s so-called persuasion model, which generates a score predicting, from zero to 10, the likelihood that a voter can be pushed in Obama’s direction. (The score also integrates a voter’s likelihood of casting a ballot altogether, so that field organizers focus the attention on those with the best chances of turning out.) A zero designates a voter likely to be repelled by the interaction, and actually pushed toward Romney or a third-party candidate; a one projects a minimal possibility of persuasion; a nine someone who can be easily pushed.

Campaign strategists have traditionally been so fearful of triggering a backlash that they rarely entrust volunteers with persuasion efforts. When placed at a phone or given a clipboard to knock on doors, volunteers usually are given tasks that do not require them to discuss sensitive or complex topics—their role has typically just been asking voters who they support, and reminding those who declare their support to turn out.

(While in 2008 Obama encouraged volunteers to make the case for the Democratic candidate in their communities, the campaign never saw it as a replacement for their paid persuasion strategy. One adviser from that campaign mockingly describes the 2008 sensibility as “building this utopian society where people talk with their neighbors.” Obama’s strategists certainly didn’t let up in traditional channels, like television ads and direct mail, where they can deploy language and imagery delicately calibrated after polling and focus-group research.)

“Persuasion calls are a more difficult thing for a volunteer to do because it’s a lot easier to hang up on someone than slam a door in their face,” says Wisconsin Democratic Party chairman Mike Tate. “You’re not just asking someone who they’re going to vote for or reminding them to vote—you’re going to people who are undecided, who don’t want to hear from you, and are often sick of politics.”

Now, thanks to its experiments, the campaign feels confident enough in its ability to identify persuadable voters that it can direct well-trained volunteers to call them with pre-written scripts. (In an election year when so few voters are at all open-minded about the candidates, true persuasion targets are so dispersed that it is rarely efficient to send volunteers walking among their houses.) The messages are crafted for different kinds of persuadable voters. Obama’s persuasion message for certain female targets threatens a “return to an era when women didn’t have control over own health choices.” Analytics are transforming the role, and value, of volunteers.

Romney’s campaign, meanwhile, appears to be selecting targets largely through the same method used in George W. Bush’s 2004 campaign. After asking which candidate a respondent supports, the surveys that feed into Romney’s microtargeting models also plumb for a voter’s “anger points.” How angry does Obamacare make you? What about growing deficits? With this data, Romney’s targeters are able to model the likelihood that a voter will respond emotionally to one of its appeals—and if that person appears in the middle range between predicted support for Obama and Romney, the campaign will send a sequence of mail pieces on a related theme, like economics or social issues. While Republicans add modest pro-Romney “advocacy” messages at the top of the scripts used at their Victory Center phone banks to identify potential supporters, they are relying on paid channels, not volunteers, to deliver persuasion messages.

This summer, the Romney campaign appears to have concluded Sarah Yancey was persuadable, and then assigned her into the issue bucket she fit best. In this case, she got assigned “the economy,” which explains the series of at least 10 mailers she received about coal, welfare, deficits, and spending. “All my mail from Romney about coal seems completely irrelevant to me,” she says.

In August, Sarah moved out of her parents’ house and into an apartment complex in another Virginia county two hours to the northeast. She registered to vote there, and at the new address started receiving mail from Obama’s campaign for the first time. It was less demographically jarring: One piece dealt with birth control, another with rising education costs. Obama’s analysts clearly now saw her as one of those middle-of-the-roaders who was a good target for persuasion, and also probably concluded they had no other way to reach her besides through the mail: Unlike her parents, she has no landline at her current address, so if they had been trying to reach her by phone, or had planned to, that option was no longer available. (Obama’s campaign has experimented with individual “callability” scores, refining the ability to predict how easy it would be to reach a given voter by phone.) Sarah says she has been planning to vote for Obama all year, and has never revisited that choice, though the campaign’s contact has successfully warded off any ambivalence. “The mailings I’ve received have made me more enthusiastic about my choice,” she says. The campaign hadn’t converted a new vote, but it successfully shored up an existing one.

Targeting is by its nature a game of imperfect predictions. Campaigns want to sort voters into different buckets so that they can design the most meaningful possible interactions possible based on incomplete, inconsistent, and uneven data. Targeters know they will always miss their marks, so their goal is to intelligently assess that risk in a way that allows them to minimize the costs (economic and electoral) of misfiring. Democrats have gotten smarter about acknowledging the limitations of solely directing persuasion efforts to the middle part of the spectrum, and are ending this election season with a major advantage in managing risk as they set out to engage the few votes whose minds can still be won. They are also able to confidently extend their hunt for persuadable voters outside the unexpectedly perilous middle terrain and to calculate who among them will be responsive to particular messages (like on Medicare) or specific modes of contact (a call from a volunteer).

Yet while Democrats may be using experiments to expand their universe of persuadable voters, they acknowledge that they thus far lack the ability to exclude those, like Sarah, who may not have ever been truly persuadable in the first place. Democratic analyst Tom Bonier says one of the post-election priorities of his Clarity Campaign Labs will be research and testing to “differentiate the mid-partisans who are there due to lack of information versus those who are there due to conflicting or counter-pressured data.” Already, according to Bonier, the new firm has started randomly assigning treatment and control messages to its surveys in the hopes of “beginning to paint a picture of potential movers to tease out the mid-partisans.”

For its part, the Romney campaign has still not given up on persuading Sarah, but appears to have failed a more basic test of tracking voter behavior. The campaign is still sending her mail at an address where she no longer lives or votes.


The Air War: Team Obama aired about 5,000 more ads than Team Romney last week

October 18, 2012

Romney’s strong fundraising and big spending suggest that he and allied groups might finally eliminate Obama’s advertising advantage.  But as of the week ending October 14, that has not happened. As the graph below indicates, advertising spending on behalf of Obama continues to outpace spending on behalf of Romney.  Obama and allies aired about 5,000 more television ads than Romney and allies last week.

Why is this happening?  One of the challenges facing Romney and Republican-affiliated super-PACs and 501c’s is that they are currently paying higher rates for advertising than Obama.  This reflects both campaign finance law, which allows candidates to pay lower rates than independent groups, as well as Romney’s decision to buy advertising time relatively late in the cycle.

Here is what this means: across all the presidential general election advertising by candidates, parties, and groups, each pro-Obama ad has cost an average of $502 dollars.  But each pro-Romney ad has cost an average of $630.

This may also help to explain why Obama has retained an edge in most battleground states.  Here is how both campaigns were targeting their ads in the previous week:

Both Romney and Obama are pursuing largely the same targeting strategy, focusing on Florida, Ohio, and Virginia.  The next tier of states is Nevada, Colorado, Iowa, and—despite some reports to the contrary—North Carolina.  Obama’s spending has outpaced Romney’s most significantly in Nevada and Florida.  The Obama campaign’s focus on Florida is interesting and, ultimately, perhaps fruitless.  Of the 5 toss-up states on the Pollster map, Florida is currently the state where Obama has the smallest chance of winning.  Nate Silver also sees Florida as an unlikely tipping point in this election.

Is any of this advertising making a difference?  That is the ultimate question and one I will revisit.  For the moment, however, there is one study worth noting.  The consulting firm Evolving Strategies recently completed a large randomized experiment in which participants saw pro-Romney ads, pro-Obama ads, both, or neither.  On the whole, the Obama ads were more effective in persuading weak partisans and undecided voters—even when the Romney ads were shown alongside.  Their effect was particularly notable among women.

However, there was a potentially countervailing effect as well: these ads tend to increase the enthusiasm of Republican voters but not Democratic voters, which could translate into additional Republican turnout.

I’ll be back next week to update these numbers.

John Sides is a professor of political science at George Washington University and a founding member of the Monkey Cage.

Obama’s Ads Are Working; Romney’s Aren’t

October 17, 2012

A new study finds swing voters are persuaded by the president’s commercials — while his challenger’s spots fail to move the needle.

Day in and day out, in battleground states across the country, voters are seeing ads like the one above. The messages they’re absorbing from this advertising onslaught have an enormous impact, relatively speaking. Yet while the candidates’ speeches on the campaign trail are covered and dissected exhaustively, the impact of the ads is far less examined, as it’s almost impossible for reporters to gauge the strategy behind their dissemination and the role they’re playing in candidates’ fortunes. A fascinating recent Politico story did much to reveal the different Romney and Obama ad-buying strategies, but the content and effectiveness of the ads remains difficult to evaluate.

new study aims to bring some clarity to that muddle. A market-research firm called Qualtrics, working with public-opinion shop Evolving Strategies, did a controlled experiment testing the reactions of independent and persuadable voters to ads from Romney, Obama, and a Republican super PAC. They found that Obama’s ads were working to sway swing voters, while Romney’s were not — and the Koch Brothers-backed GOP super PAC, Americans for Prosperity, didn’t help Romney, either.

The study exposed 2,300 voters to Romney and Obama ads on three themes —  Medicare, economic plans, and economy-based attacks on the other candidate —  as well as the Americans for Prosperity ad, “Disappointed.” A control group didn’t see any ads. All the respondents were either pure independents or weak partisans; none were strong Democrats or Republicans.

Obama’s ads overall had the desired effect: They increased his share of the vote by 6 percentage points while decreasing Romney’s share of the vote by 8 points on average. Romney’s ads, meanwhile, had no statistically significant effect on the survey respondents. The survey sample began the experiment favoring Romney over Obama, 47 percent to 42 percent; after watching both candidates’ ads, they favored Obama, 48 percent to 41 percent.

There was a silver lining for Romney, however. His ads didn’t convert swing voters, but they did persuade voters who picked John McCain in 2008 to vote for Romney this time around. Obama’s ads had no impact on his supporters’ enthusiasm. After watching both candidates’ ads, the percentage of McCain voters extremely enthusiastic about voting increased 13 points, from 31 percent to 44 percent, while extremely enthusiastic Obama voters held steady at 38 percent. That means Romney’s ads could be doing him some good by firing up his partisans so that they don’t stay home on Election Day.

As for the super PAC, with friends like these, Romney may not need enemies. The Americans for Prosperity ad features testimonials from Obama voters who say the president has let them down. The study found it had no effect on the vote overall and actually hurt Romney with women voters. The only positive effect of the ad was a large increase in enthusiasm among males who voted for McCain in 2008. “Surprisingly, the ‘Disappointed’ ad is terrible as a soft-edged appeal to swing voters, but seems to be very effective red meat for male voters in Romney’s base,” the study notes.

Obama’s Ads Are Working; Romney’s, Not So Much

October 16, 2012

A new study finds swing voters are persuaded by the president’s commercials — while his challenger’s spots fail to move the needle.

Day in and day out, in battleground states across the country, voters are seeing ads like the one above. The messages they’re absorbing from this advertising onslaught have an enormous impact, relatively speaking. Yet while the candidates’ speeches on the campaign trail are covered and dissected exhaustively, the impact of the ads is far less examined, as it’s almost impossible for reporters to gauge the strategy behind their dissemination and the role they’re playing in candidates’ fortunes. Afascinating recent Politico story did much to reveal the different Romney and Obama ad-buying strategies, but the content and effectiveness of the ads remains difficult to evaluate.

new study aims to bring some clarity to that muddle. A market-research firm called Qualtrics, working with public-opinion shop Evolving Strategies, did a controlled experiment testing the reactions of independent and persuadable voters to ads from Romney, Obama, and a Republican super PAC. They found that Obama’s ads were working to sway swing voters, while Romney’s were not — and the Koch Brothers-backed GOP super PAC, Americans for Prosperity, didn’t help Romney either.

The study exposed 2,300 voters to Romney and Obama ads on three themes — Medicare, economic plans, and economy-based attacks on the other candidate — as well as the Americans for Prosperity ad, “Disappointed.” A control group didn’t see any ads. All the respondents were either pure independents or weak partisans; none were strong Democrats or Republicans.

Obama’s ads overall had the desired effect: They increased his share of the vote by six points while decreasing Romney’s share of the vote by 8 points on average. Romney’s ads, meanwhile, had no statistically significant effect on the survey respondents. The survey sample began the experiment favoring Romney by a 47-42 margin; after watching both candidates’ ads, they favored Obama, 48-41.

There was a silver lining for Romney, however. His ads didn’t convert swing voters, but they did persuade voters who picked John McCain in 2008 to vote for Romney this time around. Obama’s ads had no impact on his supporters’ enthusiasm. After watching both candidates’ ads, the percentage of McCain voters extremely enthusiastic about voting increased 13 points, from 31 percent to 44 percent, while extremely enthusiastic Obama voters held steady at 38 percent. That means Romney’s ads could be doing him some good by firing up his partisans so that they don’t stay home on Election Day.

As for the super PAC, with friends like these, Romney may not need enemies. The Americans for Prosperity ad features testimonials from Obama voters who say the president has let them down. The study found it had no effect on the vote overall and actually hurt Romney with women voters. The only positive effect of the ad was a large increase in enthusiasm among males who voted for McCain in 2008. “Surprisingly, the ‘Disappointed’ ad is terrible as a soft-edged appeal to swing voters, but seems to be very effective red meat for male voters in Romney’s base,” the study notes.

If Obama Is Bouncing, Which Voters Are Moving?

September 10, 2012

So voters finally seem to be moving, part of what’s being called an Obama convention bounce. But who exactly is doing the moving?

I recently wrote about one of the “PocketTrial” lab experiments run by Adam Schaeffer of the Republican opinion-research firm Evolving Strategies.  Schaeffer randomly assigned an online sample of voters to watch either a Romney or an Obama campaign video, and then attributed change in each candidate’s support to the video’s influence.

The most interesting finding from the experiment was that male viewers were more easily susceptible to persuasion than female ones, shifting their opinion in response to both ads while women remained relatively stable. “A larger portion of men are decided, but the proportion that are conflicted are more variable,” Schaeffer says.

Schaeffer then looked at another dataset to see if it showed the same gender split.  He looked at the last eight samples from the Cooperative Campaign Analysis Project, for which the pollster YouGov surveys 1,000 respondents in weekly wave, stretching back from early July to just before the conventions. Each time, between 200 and 400 voters in the sample did not identify strongly with a party.

Schaeffer split that sub-sample by gender, and calculated the average share of undecided voters—in YouGov’s polls they’re categorized “not sure”—across the eight-week period.  He then looked at how much the number of undecided fluctuated week to week, by comparing the average to the wave in which it was highest and the wave it was lowest.  Among women, the mean “not sure” was 26 percent, an average between a minimum of 22 percent one week and 32 percent another.  Fewer of the men were undecided, but they swung more, from a minimum of 8 percent to a maximum of 22 percent around a mean of 15 percent. (In statistical terms, that means the number of standard deviations from the mean is 46 percent higher among men than women.)

The findings that the independent male vote is more volatile raises few possibilities.  Men could be moving more in this election, as Schaeffer’s lab experiment suggested: they’re more susceptible to persuasive messages for and against candidates.  But there could be a behavioral explanation, as well: what if men are more ready to commit to attach themselves to a new opinion after forming it—like, say, if inspired by by a welll-executed convention—and women are more tentative about making such a commitment?

Do People Have To Like an Ad for It To Work?

August 24, 2012

Aug. 24, 2012

After musing this morning on different ways of thinking about what a TV ad can do, I’d be remiss if I didn’t note two sources that have spent the summer auditioning new tools to assess the success of a single spot.

Vanderbilt University political scientist John Geer has teamed up with YouGov, a pollster that uses Internet-based samples, and launched the Ad Rating Project, which allows a 600-person panel to judge ads. The project is designed to be a democratized counterweight to punditry. “We no longer have to rely solely on ad-hoc conversations among experts, various fact checkers, and journalists about whether an ad crosses the line,” Geer and YouGov’s Doug Rivers wrote in Politico. “We instead give Americans a chance to weigh in with their thoughts.”  As a consequence, it relies on purely qualitative questions. Did viewers like the ad? Was it memorable, interesting, unfair, untruthful or unbelievable? Did it make them feel hopeful, happy, disgusted, angry, and worried?

These are, of course, foremost judgments about aesthetics and propriety. Adam Schaeffer, a co-founder and research director of the Republican consulting firm Evolving Strategies, is trying to measure impact.

His firm is promoting the use of what it brands a “PocketTrial,” basically a lab experiment, in which an online sample is randomly shown ads and then polled on the state of the race and candidate popularity afterwards. Since the viewers were assigned to groups randomly, Schaeffer can attribute differences in how the groups respond to the influence of the ads they saw. “This is the only way you can get some purchase on causality,” he says.

When he recently showed two short videos to his subjects to assess the impact of messaging related to Paul Ryan’s selection—one from Romney’s campaign trumpeting the ticket, the other from Obama’s attacking the Ryan Plan—in each case male viewers moved dramatically in the directions each ad wanted to push them while female ones barely budged. “Women seem more stable in their vote choice,” he says.

Schaeffer is reassuringly timid about venturing an assumption as to why that might be. But he scoffs at the idea that the answer could be uncovered by asking voters to declare whether they find an ad interesting or claim to be disgusted by it. “We just observe—we don’t ask people to judge an ad,” he says. “We don’t care what they think of it.”