with: Sawood Alam, Mohamed Aturban, John Berlin, Kritika Garg, Hussam Hallak, Himarsha Jayanetti, Mat Kelly, Valentina Neblitt-Jones, Michele C. Weigle
Web Archiving / Ukraine Accountability
Starling Lab
2022-08-25
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
2
Risk:
Public perception & confusion
of who and what is a web archive
#Disclaimer: “…both the live Web and the Wayback Machine [...] are reasonably reliable for everyday use”
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Screenshots are not archives,
they are the potato chips of evidence
3
Obviously, this is a faked screenshot: among other things, Starling Lab did not exist 10 years ago.
Screenshots are like potato chips in that they can enhance your meal and having a few occasionally won't hurt you, but your diet cannot solely rely on them.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
4
Web archiving is not file backup
Backup = prevent, detect, repair changes
Web archiving = continuous change to better simulate the past
Web archiving is a simulacrum of the past
https://makeagif.com/gif/blade-runner-jf-sebastians-toys-kaiser-and-bear-AFkWpp
https://ws-dl.blogspot.com/2020/03/2020-03-07-at-nexus-of-cni-keynote-and.html
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Internet Archive:
the Walter Cronkite of web archives?
5
https://www.britannica.com/biography/Walter-Cronkite
https://medium.com/tvnewsanalyzer/visualizing-the-who-and-what-of-cable-tv-news-f51d314b4c2d
Cable news now offers greater diversity, representation, and POV. However, few anchors offer the gravitas of “Uncle Walter”, “the most trusted man in America”, and some intentionally deceive.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
6
"The person with an archive knows what the page looked like.
The person with two archives is never sure."
– Segal’s Law, restated for web archives
"Mo Archives, Mo Problems"
– Notorious B.I.G. on web archives
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
There are many public web archives, each with different collection development policies
7
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Archives are not neutral
8
Including or excluding certain URLs from your archive is a choice.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Excluded from IA,
replayable from other web archives
9
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Two basic kinds of public web archives
10
Page-at-a-time
Continuous crawl
There are subtle but important differences in their operation, capabilities, evaluation, and associated risks.
It is confounded by the fact that some archives do both.
"This page is important and I want to preserve it as is for the next 10 years."
"I just learned of this page; I wonder what it looked like 10 years ago?"
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
We estimated that ~2/3 of web traffic
is not publicly archivable
11
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Tools for archiving the private web exist,
but the practice, at least as we might think of it,
is not yet widespread
12
https://oduwsdl.github.io/nehdhig2017/
https://ws-dl.blogspot.com/2019/09/2019-09-02-so-long-and-thanks-for-all.html
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
13
Risk:
Contraband or restricted pages in the archive
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Can I "swat" your archive
by having it crawl certain pages?
14
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Or simply borrow the "halo" of your archive to host my disinformation?
15
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Restricted pages in internal archives
16
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
17
Risk:
Hacking into the archives and/or insider threats
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Hacking into the archive or insider threats:
both real, with the latter more likely,
but both are outside the scope of our talk
18
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Three copies archived at exactly the same time -- What are the chances?!
Actually, there are three copies of the same observation, not three independent observations.
19
$ curl -iLs memgator.cs.odu.edu/timemap/link/https://blog.reidreport.com | grep 20051213063757
<https://webarchive.loc.gov/all/20051213063757/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT",
<http://archive.md/20051213063757/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT",
<https://web.archive.org/web/20051213063757/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT",
It will never be 2005 again, so hosting IA’s WARC files from 2005 is the best we can do.
Going forward, it would be nice to have 3+ independent observations, which could all be different because of GeoIP, personalization, CDN status, etc.
Then it’s up to the reader to determine if the differences
are semantically meaningful.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Three copies archived at exactly the same time -- What are the chances?!
Actually, there are three copies of the same observation, not three independent observations.
20
$ curl -iLs memgator.cs.odu.edu/timemap/link/https://blog.reidreport.com | grep 20051213063757
<https://webarchive.loc.gov/all/20051213063757/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT",
<http://archive.md/20051213063757/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT",
<https://web.archive.org/web/20051213063757/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT",
It will never be 2005 again, so hosting IA’s WARC files from 2005 is the best we can do.
Going forward, it would be nice to have 3+ independent observations, which could all be different because of GeoIP, personalization, CDN status, etc.
Then it’s up to the reader to determine if the differences
are semantically meaningful.
this means we have 3 different copies
of the same image …
…not three independent images taken
at (approximately) the same time
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
21
Risk:
Malicious pages designed to attack the archive, its UI, create illusions, etc.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
2017: First published works about
robustness vs. malicious .html/.js?
22
http://labs.rhizome.org/presentations/security.html#/
https://blog.dshr.org/2017/06/wac2017-security-issues-for-web-archives.html
https://acmccs.github.io/papers/p1741-lernerAT3.pdf
https://blog.dshr.org/2017/09/attacking-users-of-wayback-machine.html
Prior to these works, our group (@WebSciDL) had observed: Zombies (live web leakage into the archive), Temporal Violations (replaying web pages that never existed), Cookie Violations, Twitter replay problems, etc., but we never considered ingesting malicious .html/.js until these groundbreaking pubs.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
2018: Web IDL & Client-side rewriting
2020: Analysis of attacks on rehosting sites
23
I signed off on John’s thesis 4 years ago, but I’m only now really understanding it.
Key contribution: web archives
as subclass of rehosting sites.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
It is now harder to get live web resources into IA
24
As a result of the Lerner et al. paper in 2017, the IA added a Content-Security-Policy header to prevent live web leakage.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
But most web archives still allow live web leakage (AKA "Zombies")
25
these URLs should all be from arquivo.pt
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
If you can't get files from the live web…
…get them from elsewhere in the archive
26
``...the chronoscope is inherently limited to recent times—but what if, instead of focusing it upon the past of a generation earlier, it were tuned to the past of one-hundredth of a second ago? The dead past, Araman says, is only a synonym for "the living present". ''
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Temporal spread, temporal violations
27
https://twitter.com/phonedude_mln/status/990054945457147904
https://ws-dl.blogspot.com/2015/12/2015-12-08-evaluating-temporal.html
A large temporal spread, but not necessarily a temporal violation.
This is the semantically meaningful temporal violation
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Youtube video of Brian Williams rapping "Gin & Juice" archived in the Wayback Machine in 1992
28
John Berlin, MS Thesis, 2018
https://www.youtube.com/watch?v=k3QTcJZdFfs
(actual URI-R & URI-M have also been obscured in the video to hide the technique)
Everything in the above statement is obviously false, but the page gives the illusion of being archived in 1992.
Due to improvements in the IA playback, this illusion no longer works. Similar illusions are still possible, but are harder to construct.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
The tradespace of functionality & security
29
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
30
Risk:
"Normal" glitches & quirks of crawling and playback undermine user expectation and
can be indistinguishable from tampering
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Twitter template text isn’t
always in the expected language
31
https://ws-dl.blogspot.com/2018/03/2018-03-21-cookies-are-why-your.html
https://blog.dshr.org/2018/04/all-your-tweets-are-belong-to-kannada.html
This page is not wrong -- this content did exist on the live web at this URL for some people -- but for us it is unexpected.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
We can replay pages that
never existed on the live web
32
This page is wrong: no one on the live web ever saw a English/Portuguese/Urdu combo page.
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Load the archived page, get an eagle
33
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Hit “reload”, get a tiger
34
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Hit “reload” again, get a mountain
35
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Randomness is caused by JavaScript
As long as all three images are archived, all three valid states for this page can be replayed
36
The HTML remains unchanged, but the composite page changes randomly.
https://ws-dl.blogspot.com/2017/12/2017-12-11-difficulties-in-timestamping.html
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
The fws.gov example is easy to diagnose;
most changes are much harder to explain
37
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
2022 replay: less is more?
38
The inaccurate weather is now missing.
Is that better? Probably?
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
“No man ever steps in the same river twice, for it's not the same river and he's not the same man”
39
For third party playback, we are far from being able to do meaningful audits: replaying the same archived page over and over produces different results.
Left: Reload 1566 archived pages 39 times over 1 year.
Green=resource loaded,
Gray = resource not loaded,
Black line = baseline download.
https://github.com/oduwsdl/mementos-fixity
Conventional fixity-based approaches will not work.
We can’t depend on the archive for fixity; archives change and/or die.
Cf. “Where did the archive go?”
“Archive Assisted Archival Fixity Verification Framework”
https://arxiv.org/abs/1905.12565
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Some pages change on every single replay
40
$ curl -ks "https://wr.perma-archives.org/public/3phb-8lm6/20190818173517mhttp://web.archive.org/web/20161107234222/https:/twitter.com/GenFlynn/status/794000841518776320" | md5
af17fbd9c3b724d5b2e194d02cd1b2d0
$ curl -ks "https://wr.perma-archives.org/public/3phb-8lm6/20190818173517mp_/http://web.archive.org/web/20161107234222/https:/twitter.com/GenFlynn/status/794000841518776320" | md5
af17fbd9c3b724d5b2e194d02cd1b2d0
$ curl -ks "https://wr.perma-archives.org/public/3phb-8lm6/20190818173517mp_/http://web.archive.org/web/20161107234222/https:/twitter.com/GenFlynn/status/794000841518776320" | md5
af17fbd9c3b724d5b2e194d02cd1b2d0
$ curl -ks "https://wr.perma-archives.org/public/3phb-8lm6/20190818173517mp_/http://web.archive.org/web/20161107234222/https:/twitter.com/GenFlynn/status/794000841518776320" | md5
af17fbd9c3b724d5b2e194d02cd1b2d0
$ curl -ks "https://wr.perma-archives.org/public/3phb-8lm6/20190818173517mp_/http://web.archive.org/web/20161107234222/https:/twitter.com/GenFlynn/status/794000841518776320" | md5
af17fbd9c3b724d5b2e194d02cd1b2d0
$ curl -ks "https://wr.perma-archives.org/public/r84b-ljwj/20190613150656mp_/http://csrc.nist.gov/" | md5
4e39ba244179d4d83ca5ffb3e4ec57d2
$ curl -ks "https://wr.perma-archives.org/public/r84b-ljwj/20190613150656mp_/http://csrc.nist.gov/" | md5
d7d0cc09dc94899171604274f9b95a80
$ curl -ks "https://wr.perma-archives.org/public/r84b-ljwj/20190613150656mp_/http://csrc.nist.gov/" | md5
b7445e36e4886b6bfaa98038f87f2808
$ curl -ks "https://wr.perma-archives.org/public/r84b-ljwj/20190613150656mp_/http://csrc.nist.gov/" | md5
2f45763a7b9ef28dc440ebff469703d0
$ curl -ks "https://wr.perma-archives.org/public/r84b-ljwj/20190613150656mp_/http://csrc.nist.gov/" | md5
1fc2381b76795d31aa7c28cea7236dbe
This is expected behavior: grab the same HTML, get the same hash
This is unexpected behavior: a different hash for the same HTML every single time
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Abstracted web archive stack
or: verifying fixity is difficult because archives are active simulations of the past
41
Network Infrastructure
HTTP server
Replay Engine
Archived Resources
replayed page on your machine
Cloudflare, Akamai
Apache, nginx
Open Wayback, pywb, replayweb.page
WARC, WACZ, HAR
The resources at this level should not change!
The services at
these
levels
change
all
the
time
What you experience at this level depends on everything below, which means it will change!
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
The simulation of the past is
constantly being improved
42
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Irony: my laptop is no longer new enough
to replay this 3 year old page
The latest playback engine does not run on my ancient laptop
43
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Fortunately "legacy" mode is available
44
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Transient errors still happen
45
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
46
Risk:
Non-interoperable, non-cooperating archives
"I can archive any page* on the web!"
*except a page from another web archive
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
That archives don’t ingest the output of other archives
is a lack of interoperability.
That we’re not more concerned about this is a lack of cooperation.
47
https://www.slideshare.net/phonedude/web-archives-at-the-nexus-of-good-fakes-and-flawed-originals/87
1
2
3
4
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Archiving the archive, via the replay interface,
is critical because that's what users see
48
replay of page, archived on August 18, 2020, https://web.archive.org/web/20200818055223/https://twitter.com/realdonaldtrump
in November, 2020 (left) and August, 2022 (right)
The tweets and "What's happening" sections are different,
the media and "You might like" sections are the same.
What will this archived page look like when you click on it?
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Kudos to archive.today for preserving machine-readable
source metadata and including it in the UI
49
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL
Summary
50
Starling Lab 2022-08-25 @phonedude_mln, @WebSciDL