Proposed Effects Of Participation In Creation Of Political Persuasion Material On Subjects’ Willingness To Critically Evaluate Preference-consistent Information

Luke Zaccaro

Michigan State University MI 831

Instructor: Patrick Shaw


The relationship of politics to confirmation bias, the Dunning-Kruger effect, and the false consensus effect have been well studied, as have possible ways to overcome these issues. However, no study has been conducted to measure the effects of a game that directly helps educate a player on how these processes are perpetuated and exacerbated. This study proposes to engage players in a game where they are responsible for creating partisan information that will be disseminated to a simulated population of voters. The player will be guided on how to use confirmation bias, Dunning-Kruger and false consensus effects to engage in partisan persuasion. Our hypothesis is that the player, after becoming familiar with some of the methods and mechanics of political persuasion materials, will be more willing to critically evaluate preference-consistent partisan material.


Citizens of a successful democracy need to be intellectually capable of holding elected leaders accountable for their actions. In order to do this, they need to be able to view information and facts in a critical and rational way. If one political party’s supporters do not connect that party’s actions with the consequences of those actions because they believe incorrect information, then that party has ceased to be accountable to its supporters. That party, then, is no longer participating in a democracy.

Belief in easily disproven falsehoods is extremely and disturbingly widespread. Public Policy Polling results show that 37% of Americans believe global warming is a hoax, 20% believe that there is a link between childhood vaccines and autism, 15% believe that the government adds mind-controlling technology to TV signals, and 13% believe that President Obama is the anti-Christ (with another 13% “not sure”) (Public Policy Polling, 2013).

It is well known in the world of politics that it is difficult to persuade someone to change a strongly-held belief. People tend to filter factual information in accordance with what they already believe (Dieman-Yauman, 2011; Hernandez, 2013; Tworzecki, 2014). As a result, opinion tends inform a voter’s view of facts, rather than the other way around. This result is particularly strong when a voter believes that they have extensive knowledge of a particular issue (Hernandez, 2013). It has been demonstrated that explicitly presenting evidence to study participants that contrasts with their current viewpoint(s) does encourage a more well-rounded understanding of the topic, but people do not tend to seek out such contrasting views on their own (Vydiswaran, 2015). With modern communication technology, and the ease of avoiding preference-inconsistent information, it is therefore practically impossible to get a person to change a political belief by simply giving them information that does not reinforce their current opinion. Three of the forces that drive this phenomena are well known to psychologists. They are:

Confirmation Bias: The tendency for people to accept information that conforms to their current mental models, and to disregard information that does not. The effect is more pronounced for individuals with high levels of knowledge and/or opinion (Vydiswaran, 2015). Some studies have shown that this effect can be somewhat disrupted by presenting information in a disfluent format, such as a less clear or smaller font (Hernandez, 2013).

Dunning-Kruger Effect: The tendency for low-skilled individuals to believe themselves to be highly skilled. The abilities that allow one to be highly skilled are also the abilities one must have to accurately self-assess their own skill level (Kruger, 1999) One possible way to mitigate Dunning-Kruger, at least in school children, is to encourage belief in a growth mindset for traits like intelligence, rather than a fixed mindset. A person who believes that intelligence is malleable might make more accurate self-assessments about their skill levels (Erlinger, 2008). Viewing traits as malleable might allow a person to “separate” their identity from their traits somewhat, and thus view low skill levels as less permanent and therefore less threatening to their self-image (Rivkin, 2012). Presenting material in a disfluent format might also help to mitigate this effect, since the greater effort needed to decode information can serve as a cue that the reader has not mastered the material (Dieman-Yauman, 2011).

False Consensus Effect: The tendency of individuals to overestimate the popularity of their own opinion. The effect tends to be more pronounced for more strongly held views (Wojcieszak, 2009).

While study has been done on various ways to successfully mitigate some of these effects, that success has generally been dependent on explicitly giving participants information that is inconsistent with their current preference (Schwind, 2012; Vydiswaran, 2015). It is rare that people encounter, much less seek out, such preference-inconsistent information on their own (Vydiswaran, 2015).

The purpose of this study is not to directly measure any effect on confirmation bias, Dunning-Kruger, or the false consensus effect. The purpose of this study is to measure the effect of playing an educational game on a player’s willingness to critically evaluate preference-consistent material.

Research Question

What are the effects of playing a game where the player assumes the role of political manipulator on that player’s willingness to critically evaluate preference-consistent information?


It is almost impossible to persuade someone to change a political opinion, since there is a positive feedback loop between the strength of that opinion and the unwillingness of its owner to seek out conflicting information (Tworzecki, 2014). However, it might be possible to get them to at least critically evaluate the format in which they receive such preference-consistent information. It is thought that if a player was educated in how political persuasion materials are created with the intent to make use of confirmation bias, Dunning-Kruger, and/or the false consensus effect, that player would be more likely to recognize them in the preference-consistent materials that they encounter in their lives, and therefore more critically evaluate the source.

H1: Study participants in the experimental group will evaluate fluent, simple, and emotional preference-consistent political persuasion materials as more biased, more manipulative, more inflammatory, less informative, less useful, less true and less popular than particpants in the control group.


The game

Players will be presented with a map of a fictional country with 5 regions. The regions will have different political makeups and attributes on a handful of issues. The issues presented will not be recreations of, or even close analogues to, current prevailing US political issues. The setting and the issues need to be fictional because prior knowledge and opinion drives the confirmation bias and its associated effects (Hernandez, 2013). It is thought that a fictional setting removes the player’s emotional attachment to the issues and allows them to focus on the format of the content delivery.

The player will assume the role of “Print Communications Director,” and will be responsible for crafting political ads that will be mailed to voters. They will have a limited virtual budget, and will be responsible for allocating resources in the most effective way they can. They can spend money in two ways - creating mailers and conducting polls. The mailers will change attributes of the simulated population, and the polls measure those changes.

Players will select from lists the elements that go into political mailers - text, (Simple or complicated, rational or emotional, etc) font, size, color, etc. They will send those mailers to the simulated audience, and can then conduct polls to measure what effects those mailers had. The effects will be consistent with what research has shown the effects of various formats of communication to be. For example, fluent (simple, clear, and readable) political persuasion material requires little processing effort on the part of the reader, and so leads to greater use of heuristic reasoning (Hernandez, 2013). This would cause the receivers of the mailers who already agree with the viewpoint of the mailer to increase their confidence in their knowledge, (though not their actual knowledge) the strength of their opinion, and consequently, their likelihood of voting based on that issue.


Participants should be diverse in their political identification, so that any varying effects the game might have on different ideologies could be measured. Participants will be recruited from multiple student-body political organizations on Michigan State University’s campus. Because the effects of confirmation bias, Dunning-Kruger, and false consensus tend to be more pronounced in individuals with high levels of political knowledge and/or opinion, (Vydiswaran, 2015; Kruger, 1999; Wojcieszak, 2009) students from these politically active populations make ideal partisan subjects. They should be more motivated to participate in a political-themed activity, and are more likely to hold the biases that this study’s results are dependent upon.

Participants will not be informed of the true goal of the study. Rather, they will be told that this is a study documenting the ways in which people communicate political ideas.


Participants will create mailers from a series of attributes consistent with the different qualities (fluency vs disfluency, simple vs complex) that have an effect on confirmation bias, Dunning-Kruger, and false consensus. For example, on a fictional political issue of “taxi-tracking,” the player’s goal could be to build support for the mandatory location tracking of licensed taxis, consistent with their fictional candidate’s position. For each mailer they distribute, they will choose qualities from the following lists, which will affect the fictional voters in the following ways:

Simple vs Complex


Example Text

Result on voters


“The government has a responsibility to create a safe transportation environment for everyone. Keeping drivers accountable to you will make sure that happens. Requirements for driving a licensed taxi should include location monitoring”

actual knowledge, percentage of support


“We need to know where they are”

percentage of support, strength of opinion, Confidence, and likelihood of voting


Actual knowledge of issue




Result on voters


Infographic detailing the number of taxis on the road, how many people use the services, miles driven, potential costs or cost savings

actual knowledge, percentage of support


Picture of dark-skinned immigrant taxi driver looking over his shoulder and down at the camera - perspective is from that of a short person (child) in the backseat of a taxi. Driver’s expression is somewhat threatening.

percentage of support, strength of opinion, Confidence, and likelihood of voting


Actual knowledge of issue

Fluent vs Disfluent



Result on Voters



percentage of support, strength of opinion, Confidence, and likelihood of voting


The people should monitor the locations of licensed taxis.


Confidence in their knowledge of the issue, strength of their opinion

The player should soon learn through the feedback they receive in the form of polls, that they will be more successful at winning an election if they explicitly harden support among people who already agree with them, increase their confidence, and ignore or even falsify actual information.

Each region of the fictional country’s voters will have 5 attributes that are affected by the mailers that the player creates. These attributes can be revealed to the player by conducting a poll. For each issue, there will be an average rating for:

  1. Percentage of people supporting your candidate’s position
  2. Confidence in their knowledge of the issue
  3. Actual knowledge of the issue
  4. Strength of their opinion
  5. Likelihood of voting based on this issue


There will be two surveys that participants will take. Prior to playing the game will be a standard partisan strength and identification survey very similar to the one at This survey will seek to identify a participant’s place on the political spectrum and the strength of their political opinion. After playing, the participant will take another survey measuring their willingness to critically evaluate preference-consistent political persuasion materials. The participant will be shown political persuasion material that they already agree with (based on the results of the initial survey they took) with varying levels of fluency and complexity, and asked to rate on a 5 point scale the following qualities of each piece.







Exemplifies popular beliefs


Participants will be randomly assigned to two groups - an experimental group (55-60% of participants) and a control group (40-45% of participants).

The experimental group will begin by taking a survey measuring partisan identification and strength. Afterward, they will play the game for 45 minutes, followed by a survey measuring their willingness to critically evaluate preference-consistent political persuasion material.

The control group will begin by taking the partisan survey followed by the survey measuring their willingness to critically evaluate preference-consistent political persuasion material.


Of interest is any statistically sizable difference between the experimental group and the control group on the survey measuring their willingness to critically evaluate preference-consistent political persuasion material. If the experimental group showed a greater willingness to critically evaluate political material with which they already agree, then it can be reasoned that playing the game caused this difference.

Also of interest will be differences, if any, between subjects of different political identification and/or partisan fervor. There has been research suggesting that there are fundamental differences in the way that liberals and conservatives react to stimuli (Hibbing, 2014). This would suggest that there could be a possible difference between liberals and conservatives in the way that they initially react to political material. Some of these differences could be gleaned from the control group. Are some groups more predisposed to critical evaluation of preference-consistent information? Will playing the game have more of an effect on one group than another?

Works Cited:

Tworzecki, H., Markowski, R. (2014) Knowledge and Partisan Bias: An Uneasy Relationship. East European Politics and Societies and Cultures. Vol 28. 10.1177/0888325414535626

Hernandez, I., Preston, J.L. (2013) Disfluency disrupts the confirmation bias. Journal of Experimental Social Psychology. Vol 49. Retrieved from:

Dieman-Yauman, C., Oppenheimer, D., Vaughan, E. (2011) Fortune favors the Bold and the Italicized: Effects of disfluency on educational outcomes. Cognition. Vol 118 Retrieved From:

Mercier, H., Spreber, D. (2011) Why do humans reason? Arguments for an argumentative theory. Behavioral and Brain Sciences. Vol 34. Retrieved from: doi:10.1017/S0140525X10000968

Vydiswaran, V. V., Zhai, C., Roth, D., & Pirolli, P. (2015). Overcoming bias to learn about controversial topics. Journal Of The Association For Information Science & Technology, 66(8), 1655-1672. doi:10.1002/asi.23274

Schwind, C., Buder, J., Cress, U., Hesse, F., (2012) Preference-inconsistent recommendations: An Effective approach for reducing confirmation bias and stimulating diverget thinking? Computers & Education. Vol 58. Retrieved from:

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134. Retrieved from

Erlinger, J., Johnson, K., Banner, M., Dunning, D., Kruger, J., (2008) Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes. Vol 105. Retrieved from:

Rivkin, D., Krajčb, M., Ortmann, A. (2012) Are the unskilled doomed to remain unaware? Journal of Economic Psychology. Vol 33. Retrieved From:

Magdalena Wojcieszak. (2009) What Underlies the False Consensus Effect? How Personal Opinion and Disagreement Affect Perception of Public Opinion. Journal of Public Opinion. 10.1093/ijpor/edp001

Hibbing, J. (2014). Differences in negativity bias underlie variations in political ideology.  BEHAVIORAL AND BRAIN SCIENCES (2014) 37, 297–350 doi:10.1017/S0140525X13001192

Public Policy Polling (2013) Democrats and Republicans differ on conspiracy theory beliefs. Available: 2013 Nov 27.