1 of 17

Supporting moderators at the Wikimedia Foundation

Apps Team & Moderator Tools

2 of 17

“Moderators”?

Users with extended rights, e.g.

Stewards
Functionaries
Administrators
Patrollers
Rollbackers

3 of 17

Supporting moderators in 2023-2024

WMF has focused a lot on growing the number of new contributors over recent years

We recognise that growing the capacity for content moderation is also important

We can’t keep adding more content and more editors without also improving the other side of the process

Improve experience of editors with extended rights: Complete improvements to four workflows that improve the experience of editors with extended rights (admins, patrollers, functionaries, and moderators of all kinds); extend their creativity; impact at least four different wikis, and meet KRs for each improvement set collaboratively with the volunteers.

4 of 17

Android: Patrolling Task

Jazmin Tanner

5 of 17

It all started with Watchlist.

6 of 17

Problem Statement

Opportunity

Empowering Mobile Users
Streamlining Vandalism Action and enable focusing on nuanced edits
Fostering Positive and Productive Community Interaction
Enhancing Accessibility and Customization for Non-Technical Users
Enhancing our detection models

We understood tools like Huggle, Twinkle and SWViewer (just to name a few) existed already. However, there were specific requests to take some of those elements and integrate them into the Wikipedia app. Based on the feedback from members of the community and intensive comparative analysis we conducted on existing patrolling tools we found the following opportunities. We had the opportunity to:

Empower mobile users: It can be challenging to have multiple apps for multiple purposes, which is why we received requests to bring a tool in the official Wikipedia app. Additionally, through our research the only tool that was found favorable with mobile only users was SWViewer
You will find that our interface surfaces quality scores to aid in determining if an edit is potentially damaging and allows filtering. Our toolbar also increases access to take swift action on a feed of recent edits even if you are in transit. More importantly we hope users of the tool are able to review more nuanced edits and provide the type of guidance that only humans can
Something that stood out when getting guidance on building our base version was the importance of it not simply gamifying reverts without feedback. Supporting folks that were trying to make a good faith edit and encouraging others that are doing great work was also important, which we hope translates in our designs
Another piece of feedback that surfaced from our research was having to create custom scripts for patrolling is a barrier for some folks. There is also the challenge that some language wikis did not feel supported with current tools. While V1 of our feature is purposely basic starting out, we are aiming to add more workflows that make sense on mobile as we get feedback about our base version of the feature
Finally, we wanted to ensure that users could provide feedback to the model we are using to detect edits that are at risk of reverts so that it can continue to be improved over time. Human feedback for data models are of the utmost importance.

7 of 17

Hypothesis

Implementing Patroller Tasks in the Wikipedia apps will increase moderator velocity across several language wikis when particularly for mobile device users. Simultaneously, this initiative will reduce the barrier to patrol for less technical experienced editors.

Secondarily, the feedback provided through the Patroller Suggested Edits tasks will refine our edit quality models.

8 of 17

Evaluating Success

Satisfaction
Engagement
Impact as compared to other tools

In the future

Education

With all of these ambitious goals, how will we evaluate success? I am glad you asked. We plan to monitor

Satisfaction, this is a major indicator. Satisfaction of people using the tool as well as those that receive engagements via the tool. There will be an indicator in the edit summary when an action is taken via the app patrolling tool.
We will monitor Engagement with the tool overall. How often are people using it and then abandoning it as well as what affordances are more commonly used in case there is a request to change things around or make the case for toolbar customization, which currently exists in the app for the article view
Impact: we will view MediaWiki history to get an idea of what impact the tool is having quantitatively on content as compared to other patrolling tools. We will also evaluate how the feedback to our data models may be improving score accuracy and the adoption and acceptance rate across language wikis
Finally education, in a future version, we hope to build a training gym for expert editors who do not have extended rights yet, so that they can get a feel for patrolling and walk them through how to request rights. That training task is something we absolutely must do in partnership with current patrollers with extended rights once we ensure the patrolling task for users with extended rights is nailed down. Some Web patrolling tools are open to users that do not have extended rights, but the ones most notable on Web requires extended rights for account creation. This was a pain point surfaced in our research that we hope to tactfully address.

9 of 17

Patrollers Tasks Plan

Aug.

2023

Oct.

2023

Dec.

2023

Plan

Scale

Release MVP

Engineer

Discuss with communities (Sysops & Rollbackers)

Build V1 based on community feedback

Release with Indonesian Wikipedia

Get feedback

Apr.

2023

Mar.

2024

Scale

V1 Available

Everywhere

Jan.

2023

Plan

Release with Spanish, Chinese and French Wikipedias

10 of 17

Patroller Task V1

Here is a demo of the first version of the tool, and would love to hear feedback from you all during Q&A. Users with Sysops and Rollback rights would access the patroller task through the Suggested Edits entry point. You will be able to filter edits, and some of these options should look familiar to you based on recent edits filters on Web. You can also easily switch between languages should have extended rights on more than one language wiki. Reporting issues and accessing user information is readily available from the list view as well as the diff view. You have the ability to create and store warning messages locally on your device in V1. In future versions we will explore adding preloaded templates based on what is commonly used and agreed upon by communities. Warning messages are posted to the user’s talk page. We’ve nested Rollback in undo based on feedback to not make it too easy to rollback. The two clicks add a visual barrier. If you want to continue to monitor an article you can add it to your watchlist. You can swipe or click the buttons at the top to move to the next edit in your list. Thank is also readily available which is helpful for users that may have gotten to reverting something before you did.

11 of 17

Saved Warning Messages

Check out the designs on our project page

12 of 17

Templates

Check out the designs on our project page

13 of 17

CONNECT WITH US.

MediaWiki:

[[Wikimedia_Apps/Team/Android/Anti_Vandalism]

MetaWiki:

[[Wikimedia_Apps/Newsletter]]

14 of 17

Moderator Tools: Automoderator

Sam Walton

15 of 17

Automoderator

Automated detection and reverts for obvious vandalism, fully configurable by each Wikimedia community.

Using new machine learning models to detect edits requiring reverting

Range of configuration options to customise behaviour

16 of 17

Learn more

MediaWiki.org

[[Moderator Tools/Automoderator]]

[[Moderator Tools/Newsletter]]

17 of 17

Potential Discussion Points

What it’s like to use the feature Edit Patrol

More welcoming messages

ORES vs Revert Risk Model
Making Edit Patrol available on your Wikipedia