1 of 24

Improving �Medical Communication Through Wikipedia

Course Faculty:

�Michael Brenner, MD, FACS�Emily Capellari, MLIS

Jean Song, MSI

2 of 24

Today's Agenda

Course objectives
Wikipedia as health communication tool
WikiProject Medicine
Course syllabus
Editing and the course dashboard
Peer review
Questions

3 of 24

Course Objectives and Competencies

Understand how to appraise, assess, and apply relevant, high-quality evidence to edit Wikipedia health articles [PBLI-et, CTD-ck]
Effectively communicate health information to the public, including patients, through Wikipedia [C-pf]
Effectively peer review Wikipedia content [PBLI-ce, PBLI-sl]
Understand how the Wikipedia platform, accessible language, and content for underrepresented groups impact information disparities [PR-cd]

4 of 24

Respect and Diversity in the Classroom

We believe that diversity in students and their perspectives are a strength and benefit.
We intend to be respectful of diversity in our teaching (course materials, activities, discussions, etc).
We expect everyone to contribute to an inclusive learning environment where diverse perspectives are recognized, respected, and appreciated.

5 of 24

What is Wikipedia

Wikipedia is "an online free content encyclopedia" �accessed by "1.7 billion unique-device visitors monthly as of November 2021. It currently has more than 58 million articles in more than 300 languages, including 6,435,006 articles in English with 118,200 active contributors." �

“Imagine a world in which every single person on the planet is given free access to the sum of all human knowledge. That's what we're doing.” - Jimmy Wales

�*Cited from https://en.wikipedia.org/wiki/Wikipedia:About - accessed 1/7/2022

When was the last time you accessed Wikipedia? It may have been more recently than you think. Technologies like Siri and Alexa often answer questions using Wikipedia's freely accessible information.

Wikipedia is used world-wide as an encyclopedia, which means it provides background information and overviews of topics. Take a look at the stats that speak to the reach of Wikipedia: 1.7 billion unique users per month, more than 300 languages, and more than 6.4 million articles in English. The number of articles in English is both impressive and an example of the language bias on Wikipedia.

https://en.wikipedia.org/wiki/Wikipedia:About - accessed 1/7/2022

"Wikipedia is an online free content encyclopedia project helping create a world in which everyone can freely share in the sum of all knowledge."

"Since its creation on January 15, 2001, Wikipedia has grown into the world's largest reference website, attracting 1.7 billion unique-device visitors monthly as of November 2021. It currently has more than fifty-eight million articles in more than 300 languages, including 6,435,006 articles in English with 118,200 active contributors in the past month."

This quote from Jimmy Wales, one of the co-founders of Wikipedia, describes their vision for the resource. A vision you will contribute to through your participation in this course.

6 of 24

Wikipedia and Health

James Heilman, "Doc James", ER physician from Canada most prolific medical content editor
Wikipedia is in the top 10 ten visited websites globally*
Source of healthcare information for physicians (50%-70%), medical students (90%), policymakers, journalists and consumers**
As of March 2017

>30,000 medical articles in English; >164,000 medical articles in other languages
viewed over 10 million times per day

https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine#Metrics

*Cited from https://en.wikipedia.org/wiki/List_of_most_visited_websites - accessed 1/18/2022; December 2021 data from Similarweb �**Shafee T, Masukume G, Kipersztok L, Das D, Häggström M, Heilman J. Evolution of Wikipedia's medical content: past, present and future. J Epidemiol Community Health. 2017;71(11):1122-1129.

In 2017, the prolific medical content editor James Heilman published an article on the evolution of Wikipedia's medical content. It stated that Wikipedia is the 5th most popular website globally and is used for healthcare information by physicians, medical students, policy makers, and consumers. The article cited research on the web behaviors of 35 junior physicians, of which 70% used Wikipedia for healthcare information. While the data is in no way representative of all physicians, it provides insights into possible physician online behaviors and use of Wikipedia.

The article also documented that as of March 2017, there were over 30,000 medical articles in English and over 160 thousand medical articles in other languages. For more current statistics, you can look at the WikiProject Medicine's metrics page which shows there are currently over 50,000 articles in the WikiProject Medicine sphere. But even with these 2017 statistics, those articles were viewed over 10 million times per day globally. That means your contributions to Wikipedia have a high chance of making a big impact!

7 of 24

Health Care Site Traffic Comparison

Page views reported by a web monitoring service. Light blue portions represent official Wikimedia Foundation data.*

*Heilman JM, West AG. Wikipedia and medicine: quantifying readership, editors, and the significance of natural language. J Med Internet Res. 2015;17(3):e62. Published 2015 Mar 4. doi:10.2196/jmir.4069

8 of 24

Who are Wikipedia Editors

There are about 42,949,309 English Wikipedia accounts and 126,898 have edited in the last 30 days
General Wikipedian editor characteristics:

87% men�13% women�1% trans�72% under the age of 40�43% enrolled in school or post-secondary education*

In 2018, 298 editors made more than 250 edits in medical articles with Doc James making 13,869 (2nd most edits)
If you make at least 10 edits, you rank in the top 5% of all Wikipedia editors

*https://en.wikipedia.org/wiki/Wikipedia:Wikipedians; accessed 1/26/22

9 of 24

Medical Students as Wikipedia Editors

"We believe that not contributing to crowd-sourced resources represents a lost opportunity for enriching medical students' learning and for disseminating more accurate, up-to-date medical information to Wikipedia's readers worldwide."

Azzam A, Bresler D, Leon A, et al. Why Medical Schools Should Embrace Wikipedia: Final-Year Medical Student Contributions to Wikipedia Articles for Academic Credit at One School. Acad Med. 2017 Feb; 92(2):194-200.

10 of 24

Wikipedia and Bias*

Variable article quality
Mostly English language
Challenging readability of articles
Insufficient numbers of editors
Overrepresentation of white, male editors from high income countries
Subjective "notability" requirements for articles

*Shafee T, Masukume G, Kipersztok L, Das D, Häggström M, Heilman J. Evolution of Wikipedia's medical content: past, present and future. J Epidemiol Community Health. 2017;71(11):1122-1129.

While Wikipedia is a freely available resource that helps reduce disparities in information access around the world, it is important to note that it has some biases and limitations. Article quality -- including depth of coverage, organization, and references -- varies widely across articles. As we discussed earlier, Wikipedia has a language bias toward English. Since Wikipedia is intended for the public, the readability -- that is vocabulary, syntax, and presentation -- of articles is important. Due to the scientific terminology and complex topics in medicine, this is an area of particular concern for articles related to health. Wikipedia has an insufficient number of editors and among those there is an overrepresentation of white, male editors from high income countries. While editors are supposed to remain neutral per Wikipedia policy, unconscious bias is still likely to affect the editors and their work. Finally, Wikipedia's notability requirements introduce systematic bias into the resource. The notability requirements are meant to avoid "indiscriminate inclusion" of topics, but they have been criticized for their subjectivity and perpetuation of inequities. Significant coverage in multiple secondary sources is more difficult to achieve for topics related to underrepresented groups, including people of color, women, and low and middle income countries.

Keep these biases and limitations in mind as you select your article and write your work plan for the course. Awareness may help you mitigate the biases in your own contributions.

11 of 24

Organizations Improving Wikipedia's Health Content

National Library of Medicine (NLM), world's largest medical library and the National Networks of Libraries of Medicine, outreach arm of NLM
Medical Schools across the globe (eg. University of California San Francisco, Icahn School of Medicine at Mount Sinai, National Taiwan University College of Medicine, Sackler Faculty of Medicine at Tel Aviv University…)
Cochrane
Cancer Research UK
Translators Without Borders
You!
Many more… (check WikiProject Medicine for more partners)

12 of 24

WikiProject Medicine

http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine

WikiProjects:

Coordinate Wikipedia improvements in specific topic areas
Assess the importance and quality of articles

WikiProject Medicine focuses on medicine and health topics

13 of 24

Finding Your Article

WikiProject Medicine Assessment statistics rank articles by:

Quality

Feature Article (FA)
Stub (useful information but limited coverage)

Importance

Top
Low

https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Assessment#Statistics

The Metrics section on the WikiProject Medicine page includes their assessment statistics that rank articles by quality and importance. The quality rating ranges from stub to feature article. Stub articles have little content, references, or structure beyond the article name, while featured articles meet high professional standards. While the importance rating goes from low to high and top priority, keep in mind that importance is a subjective quality that can introduce bias. An article could be ranked low importance if it affects a small portion of the population, but to that population the article would be highly significant.

For example, the page "HIV/AIDS in India" (https://en.wikipedia.org/wiki/Talk:HIV/AIDS_in_India) is a C quality article marked as mid importance by WikiProject Medicine. To patients, families, and physicians impacted by HIV in India, this would be highly important.

Use the quality and importance table to explore potential articles you may want to edit. Clicking on the linked numbers in the table will open up the corresponding list of articles. If you're having trouble picking an article to work on, you can explore these lists to find your article.

14 of 24

Example Stub Article

This is an example of a stub article on Wikipedia. Each article in Wikipedia has an associated talk page that you can access by click the second tab in the top left. Talk pages are discussion pages that enable editors to interact about the page, which may include plans for improvements and comments on the text.

Once you have identified an article you may want to work on for this class, take a look at the Talk page to see if there have been recent discussions about improvements. By the end of the week, you will develop your work plan and post it on the talk page of your selected article. This provides other editors interested in the topic an opportunity to engage with you about your plan and future edits. Remember, Wikipedia is a crowdsourced encyclopedia--collaborating with other Wikipedians is an important part of being a Wikipedia editor.

15 of 24

Tips & Tricks for making edits “Stick”*

Be boring.�Cite, cite, cite! �Be transparent. �Make small changes, save often.�Proceed with caution.�Make good decisions.�Be “nice”. �Be brave! �Ask questions!

*Cited from Sarah Stierch (CC By SA 3.0) and Teaching with Wikipedia: Pedagogy & Praxis presentation by Anne Cong-Huyen and Kush Patel, Digital Scholarship, University of Michigan

This is a list of tips and tricks for making your edits stick in Wikipedia. "Boring" writing covers a few of the important rules to writing for Wikipedia: write from a neutral viewpoint and use clear language and sentence structures. This helps keep the reading level appropriate for non-medical audiences.

Make sure you provide plenty of citations. Your work is more likely to be reverted if it is inadequately cited. Be transparent in the changes you are making and the descriptions you write for each edit.

As you add and remove content, make smaller changes and save often. This allows other Wikipedians to consider smaller changes separately rather than reverting larger sections of edits.

Proceed with caution by drafting in a sandbox and making good decisions. Avoid escalating disputes by assuming good intent and being nice to other editors. Entering an editing war will make it more difficult for your edits to stick.

Finally, be brave as you move forward with your edits and feel free to ask questions of the instructors, the Wikipedia expert assigned to our class, and other Wikipedians on your article's talk page.

16 of 24

Course Syllabus

Improving Medical Communication Through Wikipedia Course Syllabus
Diversity Statement
Activities and assignments are broken out for each day - time management
Different prompts are provided for each Work In Progress assignment
Grading

Now that we've discussed Wikipedia and the importance of improving health content, let's take a look at the course syllabus which will help guide your self-directed learning and contributions to Wikipedia.

We believe that diversity in students and their perspectives are a strength and benefit in this course. We intend to be respectful of diversity in our teaching, including course materials, activities, and discussions. We expect everyone to contribute to an inclusive learning environment where diverse perspectives are recognized, respected, and appreciated.

This table links our learning objectives to University of Michigan Medical School competencies, as we covered earlier in this session.

As a 50% effort 4-week elective, you are expected to engage in coursework for 80 hours over the month-long course. This works out to approximately four hours per day, five days a week. Pages 2-5 provide a detailed schedule of activities and assignments broken out by day to help with time management. Activities and assignments for each day are organized into the categories: Class Meeting (we have 5 across the month), Trainings from the Wiki Education foundation to help build your skills as a Wikipedia editor, Exercises from WikiEdu, and Assignments.

The assignments range from readings and quick tasks to article editing and reflections. Starting in the second week, we will have weekly Work in Progress meetings on Tuesday at which you will discuss the work you have completed so far, and what you plan to do the rest of the week. Each Work in Progress (WIP) assignment includes a reflection prompt. The detailed schedule is also available on the Course Dashboard timeline. By the end of Week 2, you will begin making live edits in your Wikipedia article. While you will continue to improve your article independently all month long, our peer review assignment is scheduled to start the Wednesday of Week 3. During Week 4, you will respond to the peer reviewer's comments, which may include incorporating the suggestions or, in cases of disagreement, explaining your rationale. At the course wrap up, you will provide a summary of what you accomplished, how your article compares to earlier versions, and what you learned.

Page 6 of the syllabus provides the list of expectations and requirements for passing the course. You must earn at least 8 of the 10 possible points to pass the course. Completion of this course introduction is 1 point, completion of all the WikiEdu Training tutorials is 1 point, evidence of editing your self-selected Wikipedia article is 2 points (we don't quantify how much you must edit because the amount of editing necessary may vary by article and substantive contributions aren't necessarily longer), each of the 3 Work in Progress checkpoints have 1 point for the assignment and meeting participation, and submission and participation in the course Wrap Up is 1 point.

17 of 24

Wiki Education Dashboard - Home

Wiki Education Course Dashboard
Course Activity stats and overview

18 of 24

Wiki Education Dashboard - Timeline

WikiEdu Training modules and activities

19 of 24

Wiki Education Dashboard - Articles Edited

Rating assigned by a WikiProject

Wikipedia article

Sum of words added or removed by course editors

Total page views during course

Tool to see cumulative changes

Authorship highlighting in article

References added by course editors

By Week 2 you will begin making live edits in Wikipedia. Be sure you are logged into your Wikipedia account to make sure your edits are counted. This slide shows an example of the Articles view of the Dashboard populated with a group's work. You can see the article's rating on the left which I just talked about a few slides ago, the name of the article, and some summary statistics related to work on pages by students during the course like words and references added or removed, page views, what highlighted changes an individual has made in the article, itself, and all of your cumulative changes summarized in the +/- icon. These statistics are meant to help you track your work and aren't supposed to be "judging" how "much" work you are doing as any work that you will be engaged in will be highly variable and valuable.

20 of 24

Peer Review Process - Week 3

Tuesday - Have your article ready for peer review by the end of the day
Wednesday through Friday - Complete your assigned peer review

Post peer review comments on the talk page

Week 4 - Respond to your peer review comments

Incorporate suggestions and edits in article
In the cases of disagreement, explain rationale and strive for consensus

21 of 24

Discussion: Setting Peer Review Expectations

After the discussion, the instructors will summarize the themes to guide your peer review work later in the course.

What attributes do you expect in a peer review?�
What is important to consider in the context of Wikipedia and health information?�
How do you evaluate and respond to feedback you have received in the peer review process?

In order for you to get thinking about peer-review, we wanted to briefly capture your thoughts as a cohort as to your expectations around peer review. We'll summarize the themes from our discussion today and share them back with you so you'll have them when you start peer reviewing your articles. So to start, what do you expect in a peer review - what does that mean to you?

Is what we've just discussed different in light of Wikipedia and health information? Or is it the same?

How would you evaluate and respond to feedback you've received in the peer review process?

If discussion is quiet, use TRIZ Liberating Structure (what does a bad peer review look like) as an icebreaker.

A lead section that is easy to understand�A clear structure�Balanced coverage�Neutral content�Reliable sources

22 of 24

Rubric to Guide Peer Review

23 of 24

Talk Pages

Talk pages are used for discussions between editors

Discuss plans or editing issues that arise

Articles, WikiProjects, and Users all have talk pages
Sign messages with ~~~~ to add your Wikipedia username and time stamp

24 of 24

Thank you!

Questions & Answers