What's the Deal with Mexican Exit Polls?

Some Americans and Mexicans cast ballots twice on Election Day. First, they vote in government-sponsored elections and then cast their ballots in media sponsored exit polls. For the purposes of this paper, exit polls are opinion surveys that ask voters for which candidate(s) they voted for immediately following their “exit” from the polling station. In 2004, early exit poll results had Democrat John Kerry slightly ahead of Republican George Bush in a number of battleground states that he went on to lose. The results were close enough that no media organization declared Kerry the winner in any state he went on to lose. Had Kerry won those narrow margin exit poll states, however, he would have won the presidency. Instead, George W. Bush was re-elected, and many Americans to this day believe that the exit polls were correct and fraud took place. While the merits of these fraud claims are dubious at best, exit polls have been used in Mexico to convince the public that fraud does not take place in Mexican elections. Why is there this difference? After all, American polls have been around since 1967, while Mexican exit polls struggled to gain any sort of validity until 1994. The purpose of this short blog is to investigate the differences between American and Mexican exit polls both culturally and methodologically. I will demonstrate that the Mexican exit polls used to verify elections, have been more accurate due to better statistical methods, and have never detected fraud.

The Government tries to stop exit polls in Mexico

        One of the most interesting facets of Mexican presidential elections is that only PRI (Institutional Revolutionary Party) candidates had been elected President from 1928-1994. While the party may have started off being electorally popular, more recently many Mexicans believed that the PRI only won due to electoral fraud. The question, of course, is how does one actually prove that fraud did or did not occur? One attempt to answer this question is to use exit polls to determine if any statistically significant differences between poll results and official results are observed. In 1988, the Gallup organization tried to conduct exit polls, but was shut down according to Bautista et al. The incumbent government charged that exit polling violated the Mexican Constitution. They argued that voters had the right to a secret ballot, and maintained that exit polls infringed upon this right. Extending this premise, they also kept a tight lid on pre-election polls with only 5 public polls conducted during the 1988 campaign. Of course, the explanation of constitutional violation seems disingenuous when we learn that the PRI had been conducting polls for its own benefit

By 1990, the Mexican President, sensing the distrust of the public in the Mexican electoral system, had his pollster reach out to American exit poll pioneer Warren Mitofsky to do independent exit polls. The Mexican President had seen exit polls in Mexico perform well at predicting the defeat of a PRI gubernatorial candidate in 1989. The need for accurate exit polling intensified during the 1991 midterm campaign when leading television network Televisa was finally allowed to conduct an exit poll, but was unable to project winning candidates until 3 a.m. the morning after the election. While the election was not competitive, the late release of the results caused voters once again to cry foul. Although originally uncertain, after the election Mitofsky was convinced to try his hand at exit polling in Mexican elections.

Why Exit Polls are trusted in Mexico

        Following the 1991 debacle, public exit polls had yet to declare a non-government backed challenger the winner in a major election. Party loyalists also dominated the electoral process. The 1994 Mexican electoral reforms started to eliminate the PRI’s dominance of how elections are carried out. The members of the Instituto Federal Electoral (IFE), Mexico’s board of elections, were no longer mostly presidential appointees. Instead, all three major parties selected an equal number of members. Electoral fraud became a federal offense. According to Joseph Klesner, the 1996 reforms went even further by declaring, among other things, that Mexico City’s mayor be elected (not appointed), and electoral law be “under the jurisdiction” of the Supreme Court, not the President. The 1997 Midterm elections were the first under which these would be tested.

As the government passed these reforms, Mitofsky became a respected figure in Mexico. He was clear that he worked independently of the government. His first exit polls sponsored by the Chamber of Television and Radio in the 1994 federal election were accurate to the decimal point in determining that PRI had won another election. For 1997, he teamed up with Televisa (of 1991 exit poll fame). Most pre-election polls had indicated that the PRD (Party of the Democratic Revolution) would win the 1997 Mexico City mayoral election. As the polls closed, exit polls released by Televisa showed exactly that: a large PRD victory. In 2000, Mitofsky’s polls (as well as those by other organizations) were able to predict that the PAN’s (The National Action Party) candidate, not the PRI’s, would win the presidency. In the words of Bautista, “[following the 2000 election], exit polls consolidated their role as a fixture of democratic normality in Mexico”. The rise in use of exit polls correlated well with the electoral reforms of the mid 1990’s. Exit polls were seen as verifying that PRI officials did not interfere with the electoral process in Mexico. These polls were thought of as accurate because they were able to determine the winners of the elections before the actual counting took place. More than that, those who ran the exit polls were viewed as not being biased, as they were willing to declare candidates from the three parties as winners in three different major elections. Still, none of these polls had actually prevented or forecasted fraud. The 1994 and 2000 Presidential elections, as well as the 1997 Mexico City mayoral election, were all solid (6%+) victories for the leading candidate.

Differences with American Exit Polling[1]

American elections have had little of the same drama. Polling has been freely conducted since at least the 1910s, and the first exit poll appeared in the 1967 Kentucky gubernatorial election. Since that time, exit polling has expanded to every state in the nation and is done for gubernatorial, presidential, and senatorial elections. In many elections, networks are unable to project winners even when the eventual result is a large margin for one candidate (see Arkansas 2000 among others) due to relatively (compared to Mexico) small sample sizes within one state[2] and especially low response rates (see below). Cluster sampling (i.e. taking exit polls from different precincts in one state and adding them for a state estimate) doubles to sometimes triples the normal sampling error seen in pre-election opinion polls. Mexican exit polls also suffer from cluster problems, but the sample sizes of Mexican exit polls are considerably larger because they incorporate the entire country’s voters (elections are determined by the leader of the national popular vote). Therefore, a 9% advantage in a Mexican exit poll is enough to declare a national winner, but it is not enough of a lead to forecast with sufficient certainty that a candidate has won an American state. Even on a national level, American exit polls are still less accurate than are Mexican exit polls (see below). Another difference between American and Mexican exit polls is that American candidates for office employ private pollsters, but there is no record (at least on a major scale) of American candidates hiring pollsters for exit polls. In fact, during my tenure at Pollster.com, former Democratic pollster Mark Blumenthal alerted me to the fact that the campaign pollsters’ main job is message focusing, rather than determining if a candidate will win or lose. Exit polls in America serve this purpose. Rather than just forecasting a winner, exit surveys are tools for understanding the reasons behind voters’ votes. They are extensive questionnaires networks use to fill airtime. While some exit polls in Mexico do ask similar questions, they are by no means the main purpose of them.

A Difference in Exit Poll Accuracy

        When Mitofsky visited Mexico, he discovered the most important difference between American and Mexican exit polls: response rates. One of the most commonly cited reasons that the 2004 American exit polls were incorrect was differing response rates between Democratic and Republican voters. Democratic voters were apparently more willing to talk to pollsters. In a 2004 article, Blumenthal illustrated that this Democratic skew has existed since 1990. He pointed to a study released after the 2000 election (where Florida was initially and incorrectly? called for Al Gore). Even in the blowout Presidential election of 1992, the exit polls had the same bias (2.5% in favor of the Democrat nationally) as had occurred in the 2004 election according to the Washington Post's Richard Morin. Of course, the public did not care about this difference in 1992 because Clinton won the election by a wide margin. One question that has been asked by those advancing the fraud argument in the 2004 United States election is why did those polls that showed the Kerry lead not weight against this skew? According to Blumenthal, exit pollsters simply do not weight to meet some pre-election quota (whether it be the historic shortage older voters and male voters). They do not try to assume what the actual makeup of the electorate will be, so they do not weight towards it. While this may seem like an easy problem to correct, the makeup of the electorate is not always easy to predict. The 2004 presidential electorate was more female and white than the 2008 electorate. Exit polling in Mexico has not had this problem. Response rates in Mexico do differ slightly as rural males over the age of 40 are less likely to respond. Studying four demographically diverse states in Mexico in 2004 and 2005 elections, Bautista et al. used a single proportion test to discover that these voters are slightly more likely not to support the PRI than voters who do answer the exit polls. This segment of the population is both small and the PRI has not been competitive in presidential elections since 1994. Therefore, these errors have not had a difference at determining the winner of the election.

Why Are Response Rates Different?

        Part of the reason for these differential response rates is the inferior preparation by the exit poll interviewer in the United States. In America, these interviewers are trained over the telephone, while their Mexican counterparts are trained in person. American exit pollsters are also less experienced as compared to their Mexican counterparts. According to a 2005 Edison Media/Mitofsky international report, less experienced exit pollsters during the 2004 election incurred lower response rates and, in turn, higher error rates. During the 2004 American election, many younger American exit poll interviewers decided to correct for their lack of responses by breaking from protocol. Instead of returning to interviewing every 10th voter, for example, if the previous 10th voter refused to be interviewed, interviewers tried to make up for missed voters by selecting the next willing participant (whether it be the 1st, 2nd, 3rd, etc.). Age of the interviewer also played a big role in America and Mexico. In both countries, interviewers are, on average, in their 20’s. The 2005 Edison/Mitofsky study found that many older voters in America do not want to answer exit polls given by young interviewers. This was especially a problem in 2004 when American exit pollsters employed a higher number of young interviewers, despite knowing of the past bias. And while older voters tended to vote more Republican in past elections, the difference in aggregate vote choices between age groups was historically high in 2004. In Mexico, the exact opposite was found to be the case in 2004 and 2005. Bautista et al. discovered that Mexicans were actually less likely, for whatever reason, to answer an exit poll from an older interviewer. Thus, the young age of both Mexican and American interviewers is an asset in one country (Mexico) and a detriment (America) in the other.[3]

Can Exit Polls detect Fraud in Mexico: 2006?

        While there can be little doubt that exit polls convinced the Mexican public that their elections were legitimate, no exit poll has ever proven the inverse case: an illegitimate election. During the aftermath of the 2004 American exit poll debacle, those who argued that exit polls could detect fraud pointed to Mexico as an example. Steve Freeman, a leading advocate of the “election was stolen” crowd, directed supporters to Mexico’s 2000 presidential election. Specifically, he quoted campaign strategist Dick Morris as saying

When I worked on Vicente Fox’s [PAN] campaign in Mexico, for example, I was so fearful that the governing PRI would steal the election that I had the campaign commission two U.S. firms to conduct exit polls to be released immediately after the polls closed to foreclose the possibility of finagling with the returns.

Of course, Morris never actually provided proof that the exit polls were as accurate as the final results. Neither he, nor Freeman, nor anyone else for that matter, published proof that these Mexican exit polls actually proved fraud. Exit polls coincided with the reforms of the 1990s and like American exit polls before 2000, were able to predict winners and losers. Still, there had never been a major Mexican election with a close final result and an exit poll. That all changed in 2006.

As close as the 2004 American Presidential election was, the 2006 Mexican Presidential election was even tighter. PAN’s Felipe Calderon ultimately won a very close contest against PRD’s Andres Manuel Lopez Obrador, who insisted that the election was tainted by fraud. On election night, media and campaign organizations released eight exit polls. While the individual campaign’s released polls showed their candidates ahead, the media surveys indicated a race “too close to call”. The Mexican media response to the 2006 election was the same as to the 2004 American election. These media organizations understood that any candidate lead was within the margin-of-error. The difference was that the 2004 American exit polls were leaked without an understanding of the margin-of-error or past Democratic skew. Thus, in the only major Mexican election with an exit poll and a candidate (PRD’s López Obrador) declaring fraud, the exit polls proved nothing for either side. They merely indicated that either candidate could have won. Indeed, exit polling has only hinted at, but never proven fraud in any election. Usually, such as in the Georgian and Ukrainian revolutions of the 2000’s, other more solid evidence of fraud such as eyewitness testimony, taped phone conversations, and physical evidence of vote tampering is needed to substantiate election improprieties. Exit polls are more useful as a starting point, rather than an end point, in the fraud process because they have sampling and measurement error. It is for this reason that only in cases of large vote margins (America 1980, 1984, 1988, 1992, 1996, and 2008 and Mexico 1994, 1997, and 2000) can exit polls be used to definitively declare a winner once the polls close.

What to expect in 2012?

        In less than 2 years, Americans and Mexicans will head to their voting stations to vote for president. In America, President Barack Obama will undoubtedly be re-nominated by the Democratic Party to be its presidential nominee to face off against a yet unknown Republican. In Mexico, PAN President Felipe Calderon will stand down (due to term limits), and the three main parties PAN, PRD, and PRI will nominate candidates. The early polls suggest that Obama may be in a tight fight for re-election, while the PRI seems likely to make its best showing in a Presidential election since 2000. On Election Day, exit polls will be employed to give media networks and viewers an early snapshot of who won and why. As this paper has demonstrated, Mexican exit polls have generally been more accurate than American exit surveys. American polls have tended to overestimate the Democratic candidates’ share of the vote and have not been weighted for this.

I am not sure if past trends in exit polls will hold in 2012. Mexican exit polls have shown a tendency to overstate the PRI, but the PRI has not been competitive in a presidential election since 1994. If early polls are correct about PRI competitiveness, it could lead to an election night where many exit pollsters are left wondering if the PRI’s strong showing is merely bias reappearing. In America, the Democratic skew may not materialize due to an understatement of the Mexican-American (who vote overwhelmingly Democratic) vote. As Michelson and Pallares discovered, some Mexican-Americans (a booming population in the United States) had difficulty taking the standard written American exit polls due to language barriers. These voters preferred face-to-face interviews to better understand the questions asked. If American exit polls are conducted as they have been in the past (secret ballot), it could lead to Mexican-American voters improperly filling out their ballots, invalidating the results. A publication from Latino Decisions has argued that exit polls from 2010 suffered this problem in California and Nevada. Considering these possible new problems and past exit poll difficulties, it would be wise for the media and amateurs not to jump to conclusions on election night 2012 based off of the exit polls. Instead, for both America and Mexico’s presidential elections, caution should reign and actual votes should be tallied before projections are made.

[1] For a more on American exit polling, please see the first post in this two part series.

[2] Elections in the United States are won state-by-state, and election night projections are done state-by-state.

[3] Another impacting factors could include exit poll interviewers distance from voting stations. Data suggests that in America and Mexico response rates are lower the further from a voting station an exit interviewer is locations; however, no data is available to suggest that more American interviewers are located further from voting stations.