1 of 21

THE ART OF FACT-CHECKING: �REPORTING WITH A FACT-CHECKING MINDSET

WITH ALLISON BAKER

HEAD OF RESEARCH AT THE WALRUS MAGAZINE

CO-CREATOR OF THE TRUTH IN JOURNALISM PROJECT

2 of 21

FACT-CHECKING: �WHAT IS IT, AND WHY DO WE CARE?

Fact-checking exists in various forms. It combats both fake news and bad journalism—there’s a difference.

For longform journalists (magazines, podcasts, etc.), fact-checking is developed in-house and is informed by the requirements of long-form journalism.

We think of it as an essential step of verification before publication; it is usually not visible to the outside world.

As Tom Rosenstiel and Bill Kovach wrote in The Elements of Journalism (2001), journalism is essentially a "discipline of verification," which journalists use to find not just the facts but also the "truth about the fact." A journalist (or team of journalists) reports a story, they weave their findings into a narrative, and then, before sharing it with the world, they check that what they've created stands a test of accuracy—whatever they've decided that test should be.

If fact-checking is the essence of journalism, then journalists need to learn how to do it without perpetuating inequitable standards of verification. For example: How should a fact-checker approach a story about first-person accounts of sexual violence? Or about an individual's lived experience with disability? Or about an Indigenous community's account of historical trauma? These questions have rarely been discussed explicitly or in a public-facing way, and this silence has led to serious problems in recent years, such as the infamous retraction of Rolling Stone's 2014 investigation into an alleged sexual assault on a university campus.

�

3 of 21

FACT-CHECKING: �WHAT IS IT, AND WHY DO WE CARE?

Credibility and reputation

Strengthening the story

Commitment to readers, writers, and sources

Contribution to the historical record

Legal protection

Trust and inclusion

Editorial fact-checking is important not only for the sake of journalistic accuracy but also as a tool for addressing broader issues of trust and inclusion. Reporters who operate with a problematic conception of objectivity as "the view from nowhere" (as endorsed by philosopher Thomas Nagel) often choose to distrust certain voices because of their backgrounds or experiences. They hold that someone's personal experience with a certain topic may prevent them from understanding or describing it accurately. The result of such attitudes is that, too often, the industry has failed to acknowledge the voices of marginalized communities and to treat their testimonies as credible. And this has led to what writer and producer Pacinthe Mattar calls a "crisis of credibility" in media.

Questions of sourcing and methodology in fact-checking, therefore, are intricately related to questions of equity and inclusion—including whether some voices are treated as more authoritative than others. In the post-truth era, when many have become more conscious of the fragility of "objectivity," it is important for journalists to learn how to acknowledge the close relationship between accuracy and ethical practice.

4 of 21

A FAILURE THAT WAS AVOIDABLE��“A RAPE ON CAMPUS: A BRUTAL ASSAULT AND STRUGGLE FOR JUSTICE AT UVA”�ROLLING STONE, 2014

5 of 21

“A RAPE ON CAMPUS: A BRUTAL ASSAULT AND STRUGGLE FOR JUSTICE AT UVA”�ROLLING STONE, 2014

Soon after the story's publication, the magazine was forced to effectively retract it because of factual inaccuracies; the fact-checking process had been neither thorough nor sufficient.
An independent review of the situation was conducted by the Columbia Journalism School over the following months; the reviewers found that Rolling Stone had "set aside or rationalized as unnecessary essential practices of reporting that, if pursued, would likely have led the magazine's editors to reconsider publishing [the main source's] narrative so prominently, if at all."

6 of 21

�“A RAPE ON CAMPUS: A BRUTAL ASSAULT AND STRUGGLE FOR JUSTICE AT UVA”�ROLLING STONE, 2014

“We were too deferential to our rape victim...We should have been much tougher, and in not doing that, we maybe did her a disservice….” —Rolling Stone editor
This statement implies that the editorial team had been faced with a choice: either respect their source's wishes or fact-check the story properly.
As the CJR reviewers noted, this analysis is misguide: "The explanation that Rolling Stone failed because it deferred to a victim cannot adequately account for what went wrong...The editors made judgments about attribution, fact-checking and verification that greatly increased their risks of error but had little or nothing to do with protecting [the main source's] position."

7 of 21

�“A RAPE ON CAMPUS: A BRUTAL ASSAULT AND STRUGGLE FOR JUSTICE AT UVA”�ROLLING STONE, 2014��THE REAL ISSUE:

Rolling Stone's editorial team did not ensure that necessary conversations between the reporter, fact checker, and subject of the story had taken place.

These include: what the fact-checking process would entail for the subject and others; to which verification standards the publication would hold her story; and what possible accommodations were available to her.

Essentially: they did not uphold the principles of responsible journalism, nor did they report the story with a fact-checking mindset

8 of 21

�“A RAPE ON CAMPUS: A BRUTAL ASSAULT AND STRUGGLE FOR JUSTICE AT UVA”�ROLLING STONE, 2014��FALLOUT:

Million-dollar defamation lawsuit
Loss of credibility
Harmed not only the source, but the wider conversation around campus sexual violence and believing survivors
This case shows that avoiding rigorous practices of verification may only subject a survivor of trauma to greater scrutiny, skepticism, and other harms (including lawsuits) in the future. Indeed, Rolling Stone's retraction of the story was interpreted by some as evidence for the (false) narrative that "women lie about sexual assault."

9 of 21

THE FACT-CHECKING MINDSET

ACCURACY: A statement is accurate if it is free from factual error; it reflects the "fact of the matter" about a topic, whenever such a thing exists; and it "sounds right" to the people whom the statement concerns.

TRANSPARENCY: A manner of working and communicating that makes it easy for others to understand one's methodology and motivations. In a fact-checking context, this means being open and honest about the requirements and limitations of your work and making sure that the public knows which facts have been checked and how.

INTEGRITY: Working with integrity means having clear and ethical intentions, treating others with respect and humility, and remaining aware of the context in which one is operating.

10 of 21

THE FACT-CHECKING MINDSET

Collaboration between different players who understand their respective roles ensures that journalism—and editorial fact-checking in particular—is helpful, effective, and practiced with integrity.

Key idea: journalism is a social activity, made possible by a community, including reporters, their editors, their publishers, their sources, their fact-checkers, their audience, and anyone else involved in the production of a story or affected by its content.

11 of 21

THREE COMPONENTS OF FACT-CHECKING

Verification: Given a statement and the reporter's sources for that statement, the fact checker confirms that the statement is factual, identifies errors and suggests corrections, or determines that neither of these options is possible.

Investigation: The fact checker assesses the credibility of the sources used by the reporter and finds out what other credible sources say about the statement in question. Is there any nuance or context missing from the statement? Have any sourcing considerations been overlooked or misjudged?

Documentation: After the statement is confirmed or corrected, the fact checker preserves all sources, documents, and methodology for their (and their publication's) fact-checking records.

The reporter is responsible for ensuring the accuracy of every word that appears in their piece; the fact checker should be able to defend the accuracy of every word. EMPHASIS ON COLLABORATION.

Here is a non-exhaustive list of details that should always be checked in a work of journalism:

Names, dates, and locations
Quotes and the facts within quotes
Descriptions of people, places, and events
Pronouns, identifiers, and descriptors
Causal explanations
The factual bases of opinions
The internal logic of a story, including its chronology

THE REPORTER: Should be able to account for every fact (cf. the Reporter’s Guidelines); Responsible for ensuring the article is fair and accurate before submission; Should provide a research package: contact information for sources, interview recordings and transcripts, thoroughly annotated draft; Should clearly communicate with the fact-checker about any sensitivities or off-the-record/background sources

�

12 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

1. THE TWO-LAYER PRINCIPLE

There are always two distinct steps to establishing a statement in a journalistic story: first, reporting; then, verification.

13 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

2. THE INDEPENDENCE PRINCIPLE

Every fact should be checked against at least one authoritative source or, if that is not possible, several independent non-authoritative sources.

If a fact can be verified only via non-authoritative sources, that may need to be made clear in the text of the story.

14 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

3. THE INTERAL/EXTERNAL FACT DISTINCTION

Internal facts: Facts that concern only the source in question. In principle, no other source—neither interviewed nor gathered—is better suited to check such facts.

External facts: Facts about anything other than the source to which they are attributed—including facts about external events, places, and people. In principle, any source could be mistaken about its external facts, and other sources may be more appropriate for the checking of those facts.

Every fact will be checked against at least one authoritative source or, if that is not possible, several independent nonauthoritative sources.

Most factual statements attributed to a source include a mix of internal and external facts about that source; it is the fact checker's job to parse them as best as possible and to check each kind of fact in the most appropriate way.

Internal facts: Facts that concern only the source in question. In principle, no other source—neither interviewed nor gathered—is better suited to check such facts.

External facts: Facts about anything other than the source to which they are attributed—including facts about external events, places, and people. In principle, any source could be mistaken about its external facts, and other sources may be more appropriate for the checking of those facts.

15 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

4. THE AUTHORITY PRINCIPLE

A source is always authoritative on its internal facts but only sometimes authoritative regarding its external facts, depending on the source's relationship to those facts and how they were gathered.

16 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

5. THE KNOWLEDGE PRINCIPLE

Fact-checking and reporting require one to navigate different forms of knowledge comfortably. How a fact is checked will consequently depend on the kind of fact it is and the context in which it has been reported.

For example, you should confirm a fact grounded in oral records by using oral verification methods.

17 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

6. THE PERSONAL PRINCIPLE

A fact will be confirmed with every person it involves, regardless of how you originally reported it. This means that any element of a story that concerns someone other than the person telling the story will be checked with that person too—not just for factuality but also as a principle of integrity.

18 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

7. THE COLLABORATION PRINCIPLE

All facts need to be checked before a story can be published; there cannot be a compromise on this. But the fact checker will collaborate with a source to determine how the facts pertaining to them will be checked. This includes various accommodations to ensure the accessibility of the fact-checking process.

19 of 21

FACT-CHECKING IN PRACTICE: �EIGHT GUIDING PRINCIPLES FOR REPORTING WITH A FACT-CHECKING MINDSET

8. THE POWER PRINCIPLE

Both you as the reporter and the fact checker who steps in after you should consider facts with care, think conscientiously about the relationships among people and communities, and treat all forms of knowledge as equally valid.

20 of 21

OVERVIEW

1. THE TWO-LAYER PRINCIPLE

2. THE INDEPENDENCE PRINCIPLE

3. THE INTERAL/EXTERNAL FACT DISTINCTION

4. THE AUTHORITY PRINCIPLE

5. THE KNOWLEDGE PRINCIPLE

6. THE PERSONAL PRINCIPLE

7. THE COLLABORATION PRINCIPLE

8. THE POWER PRINCIPLE

MORE IN THE TRUTH IN JOURNALISM FACT-CHECKING GUIDE:

A comprehensive, step-by-step guide to fact-checking
Trauma-informed fact-checking
Reporting on marginalized communities
Assessing authority and expertise
Identity and lived experience
Linguistic considerations
Navigating power dynamics

FIND IT HERE: https://thetijproject.ca/guide

21 of 21

QUESTIONS?

thetijproject.ca

allison@thetijproject.ca

allison@thewalrus.ca