1 of 27

OONI Hack Session

Bornhack @ CCCamp 2023

18th August 2023

2 of 27

Free software project (born under The Tor Project) aimed at empowering decentralized efforts in increasing transparency of internet censorship around the world.

Since 2012, the OONI community has collected more than a billion network measurements

from 241 countries and territories, shedding light on internet censorship worldwide.

3 of 27

You can install OONI Probe on:

iOS
Android (and F-Droid)

OONI Probe Desktop available for:

Windows
macOS
Linux

https://ooni.org/install/

https://github.com/ooni/

4 of 27

OONI Software Tests

Blocking of websites

Blocking of instant messaging apps

Blocking of censorship circumvention tools

Detection of middle boxes

Measure network speed & performance

https://ooni.org/nettest/

5 of 27

Citizen Lab

Test Lists:

Websites tested by OONI Probe

Global test list: �Internationally relevant websites�
Country-specific test lists: �Websites that are relevant to a specific country

Contribute to test lists: https://test-lists.ooni.org/�
Citizen Lab GitHub repo: https://github.com/citizenlab/test-lists

When you measure the blocking of websites with OONI Probe, which websites do you test?

OONI Probe tests websites which are included in certain lists, called “test lists”. They are called “test lists” because they are lists that include websites that are meant to be tested.

These test lists are hosted publicly by the Citizen Lab, which is a research organisation at the University of Toronto. In particular, you can find these lists on GitHub, where you can review and contribute to them.

There are 2 types of test lists:

There is the Global test list, which includes internationally relevant websites, such as facebook.com, BBC, and other websites that are commonly accessed by people worldwide. The websites that are included in the Global test list are tested by OONI Probe users in all countries around the world.
On the other hand, there are the country-specific test lists, which include websites that are only relevant to a specific country. For example, the Brazilian test list includes Brazilian news media websites, Brazilian human rights websites, and other websites that are relevant to Brazil. Depending on the country that you are in when you run OONI Probe, you will test the websites included in that country’s test list.

So overall, when you test websites with OONI Probe, you will test websites included in both the Global list and in the list that is specific to the country that you are in.

6 of 27

Test Lists Editor

https://test-lists.ooni.org/

7 of 27

Coordinate

OONI Probe

Testing

https://run.ooni.io/

8 of 27

OONI Explorer

https://explorer.ooni.org/

Through the OONI Explorer search tool, we can explore which websites are blocked around the world. To do so, access the search page of OONI Explorer. Through the search page we have various parameters based on which we can filter all of the measurements collected from around the world.

To limit our search to only those pertaining to the testing of websites, start off by selecting the options under test name to then choose the web connectivity test under websites.

Once web connectivity has been selected, we can then click filter results in order to limit our results to only those pertaining to the testing of websites. We can now see that all the results that we have are only collected from web connectivity measurements, testing various different types of websites. around the world.

Each of these rows represents a different measurement. By clicking on the row we can access the actual measurements.

9 of 27

Measurement Aggregation Toolkit

(MAT)

https://explorer.ooni.org/chart/mat

The MAT provides filters that enable you to select the parameters you care about in order to plot charts based on aggregate views of OONI data.

Specifically, the MAT includes the following filters:

* **Countries:** Select a country through the drop-down menu (the "All Countries" option will show global coverage)

* **Test Name:** Select an [OONI Probe test](https://ooni.org/nettest/) based on which you would like to get measurements (for example, select `Web Connectivity` to view the testing of websites)

* **Domain:** Type the domain for the website you would like to get measurements (e.g. `twitter.com`)

* **Website categories:** Select the [website category](https://github.com/citizenlab/test-lists/blob/master/lists/00-LEGEND-new_category_codes.csv) for which you would like to get measurements (e.g. `News Media` for news media websites)

* **ASN:** Type the [ASN](https://ooni.org/support/glossary/#asn) of the network for which you would like to get measurements (e.g. `AS30722` for Vodafone Italia)

* **Date range:** Select the date range of the measurements by adjusting the `Since` and `Until` filters

* **X axis:** Select the values that you would like to appear on the horizontal axis of your chart

* **Y axis:** Select the values that you would like to appear on the vertical axis of your chart

10 of 27

OONI Data

https://ooni.org/data/

11 of 27

OONI research reports

12 of 27

Updates from the OONI-verse

OONI Run v2�
Censorship Incident Reporting Platform

Methodology for detecting throttling

Social Media Censorship Alert System

Automating detection and characterization of censorship

13 of 27

OONI Run v2

OONI Run links will have first class support in OONI Probe

URLs will be shorter and editable

Measured as part of automatic testing and automatically updated

Results from an OONI Run link searchable in OONI Explorer

Richer API for customizing how they appear and configuring nettest options

All cards in OONI Probe dashboard will become OONI Run v2 links

14 of 27

Censorship Incident Reporting Platform

15 of 27

Methodology for detecting throttling

16 of 27

Social Media Censorship Alert System

Receive alerts on a blocking or unblocking event occurs on a particular (test_name, country, asn, input) tuple TCAI

Focus only on social media to reduce noise due to unreliability of hosting for smaller sites

RSS feed for getting the event list

17 of 27

Automating Detection and Characterization of censorship

High level goal is to be able to say

Target X is

OK - Accessible and no filtering detected
Blocked - Some form of restriction is in place
Down - The service is inaccessible, but that’s not due to blocking

More specifically for cases 2. we care to know how the block is happening

DNS tampering
TCP/IP blocking
TLS Interference
HTTP blocking
Throttling

18 of 27

Let’s go down the rabbit hole!

19 of 27

20 of 27

OONI Probe (Ideas)

OONI Probe network experiments

Bittorrent test (a PR exists, but needs more work)
Matrix messaging test
See more: https://github.com/ooni/probe/issues?q=is%3Aopen+is%3Aissue+label%3A%22new+nettest%22

OONI Probe Engine

Replace C Tor with Arti for running Tor experiments

21 of 27

OONI Data

The goal is to transform one measurement into many observations
Many observations are analyzed to determine if a target is

Blocked
Down
OK

When it’s blocked, we care to know how

22 of 27

OONI Data

Measurement

Observation

Experiment result

23 of 27

Observation generation

23

24 of 27

Observation generation

24

See WebObservation model and relevant transforms (measurement_transformer, web_connectivity)

💾

HTTP	TLS	TCP	DNS Answer	ip	hostname	probe	time
OK	OK	OK	162.19.153.247	162.19.153.247:443	quora.com	US, AS123	2023-01-01T11:01:002
OK	OK	OK	200.34.20.123	200.34.20.123:443	quora.com	US, AS123	2023-01-01T11:01:003

25 of 27

OONI Data (Ideas)

Add support for parsing unsupported experiments: https://github.com/ooni/data/issues/32
Improvements to the blocking analysis

Scale the support for ExperimentResult generation
Experiment with machine learning for DNS anomaly detection

Come up with new interesting analysis

26 of 27

OONI Explorer (Ideas)

See: https://github.com/ooni/explorer/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22
Come up with interesting new data visualizations

World map?

27 of 27

Resources