1 of 23

Saving Data Journalism��

Katherine Boss,

Meredith Broussard,

Fernando Chirigati,

Rémi Rampin,

Vicky Steeves,

New York University Libraries

NYU Arthur L. Carter Journalism Institute

NYU Tandon School of Engineering

NYU Libraries & Center for Data Science

National Institute of Computer Assisted Reporting, Newport Beach, 2019

2 of 23

How do News Apps differ from other born-digital news content?

Interactive
Exploratory
Some reliant on a database
Some use external APIs
Custom-built software

Source: Times analysis of LAFD data. Credits: Ben Welsh, Robert Lopez, Kate Linthicum. Map data: (c) OpenStreetMap contributors, CC-BY-SA. http://graphics.latimes.com/how-fast-is-lafd/#14/34.0521/-118.2756

3 of 23

The Problem:

Interactive news stories can’t be archived

4 of 23

Why?

• Web archiving crawlers were built to crawl the static web

• Current web archiving technology cannot capture dynamic elements of a site, or back-end files (like data!)

• Older interactives are dependent on older browsers to display (such as Flash-based interactives)

• Newsrooms are volatile organizations: they get bought and sold, the servers are shut down, labor is re-organized

5 of 23

Early news app from 2009 — “Fatal Flights”

The Washington Post. 2009. “Fatal Flights: Fatal Medical Helicopter Crashes.” From: http://www.washingtonpost.com/wp-srv/special/nation/medical-helicopters/fatal-crashes.html

6 of 23

ProPublica “How much is a limb worth?”

Groeger, L., Grabell, M., & Cotts, C. 2015. Workers’ Comp Benefits: How Much is a Limb Worth? From http://projects.propublica.org/graphics/workers-compensation-benefits-by-limb

7 of 23

Internet Archive snapshot of Workers’ Comp, Mar. 8 2016

8 of 23

Okay, but as long as AWS is around in 20 years, won’t my stuff be okay?

9 of 23

10 of 23

But all of my work is containerized. Isn’t Docker a reliable way of saving stuff?

11 of 23

Docker is six years old! How long will it be around? What will you do with all of those containerized interactives when Docker is gone? Archiving and preservation is also about more than just getting the files and depositing them somewhere. Proprietary software isn’t guaranteed to be interoperable in the future. It’s not a business interest to keep files forward and backwards compatible between versions. Docker 1.0 that you used to write your news app might work now, for the next 5 years, but could totally break in 10. Also, Docker itself is not stable. Last year, it didn’t work on Windows home edition and that was only recently changed! These companies make software at their own whim and make no guarantees, and so cannot be safe in the long-term.

Digital preservation is also people! It requires checks against file fixity, data integrity, information security, metadata and file formats, and more. While digital archivists can move files from Word 95 to plain text and keep a master copy, this is not possible for Docker containers.

If only there existed a cultural institution with a mandate for archiving and preserving news for the long term, and the skills to secure and preserve digital works….

12 of 23

The Solution

13 of 23

Libraries have been around for hundreds of years. In an information age libraries are changing constantly. When I tell people outside of libraries what I do for a living I occasionally get the response, “... libraries still exist?” Yes they still exist, and there is a Twitter user called “the angry librarian” who will roast you if you don’t believe libraries are thriving. They are far more likely to persist 100 years in the future than any particular proprietary software. Libraries have the mandate and the expertise to archive your stuff. So help us help you! In this Jerry Maguire analogy Libraries are Tom Cruise and you all are Cuba Gooding Jr. Help us help you archive your work. We have the mission, the infrastructure, and the people to do so. And now we might also have developed the technological tool to do so.

14 of 23

How? An emulation-based web archiving tool

NYU Libraries and the NYU Center for Data Science were awarded an IMLS grant to build an extension to ReproZip to capture remote front-end files (e.g. not located on server where app is running) to be able to capture the *complete* news app!

Our research team has built what is, to our knowledge, the first ever full stack, emulation-based web archiving tool in the world. So digital archivists have been thinking about the problems of software obsolescence for a long time, more than two decades. And these theorists agree, emulation is the only feasible solution for archiving digital works. Which means we have to capture an “object” - a video game, a CD-ROM, any sort of file, in its original computing environment and with all of its dependencies, and then emulate that environment so it can be viewed by a user in the future on a completely different computing environment.

Our emulation-based archiving tool is based on two open-source software projects: ReproZip, which is a comp reproducibility tool developed at NYU by several members of our team, and WebRecorder, which is an open source, dynamic web archiving technology developed by Rhizome. ProPublica was our partner in this work and gave us access to a number of their news applications for the research and development, want to thank Scott Klein and his team for all of their help and support in this work!

15 of 23

How does ReproZip work?

r eprozip.org & examples.reprozip.org

open, unpack, and reproduce anywhere, anytime!

necessary data files, libraries, environment variables, etc. required to reproduce your data journalism

16 of 23

ReproZip: Reproducibility in 2 Steps

Packing

Unpacking

Linux

data files, libraries,�environment variables, etc.�required to reproduce�the research

ReproZip�Bundle

Linux

macOS

Windows

open, unpack, and�reproduce anywhere, anytime!

17 of 23

Journalists

Data Analysis, App, etc.

reprozip

Executing

Tracing

Creating�Configuration

Configuration�File

Preservation

Bundle

(.rpz file)

Configuring

Packing

Input files, output files, parameters…

Data

Executable programs and steps

Workflow

Environment variables, dependencies, …

Environment

Detailed Provenance

Step 1: Tracing & packing

Computational Environment E (Linux, in this case a web server)

The basic idea of how ReproZip works is that it executes at the same time as a computational work, like an interactive! While the interactive executes and runs like it’s supposed to (start up the database, connect to server, query things, etc.), ReproZip keeps track of the workflow, provenance, input data (even databases), and all relevant environmental variables (e.g. what version of ruby are you using? it captures that information but also the source of your ruby distribution and ruby interpreter, and the source code of all gems used), and then all this information is written into an RPZ file, which is typically a very small compressed file that can be distributed with ease.

From an archiving perspective this is a very important feature - that all of the work could be contained in a single archivable and distributable file.

18 of 23

Readers

Computational Environment E’ (something potentially different from original)

Step 2: Unpacking & replaying

reprounzip

Unpacking

directory

Linux

chroot

/

vagrant

Linux�macOS�Windows

docker

Provenance�Graph

(any OS)

VisTrails

(any OS)

Linux

Linux�macOS�Windows

Preservation

Bundle

(.rpz file)

Singularity (upcoming)

ReproServer

(in-browser)

One of the best things about using ReproZip is the unpacker model for unpacking and replaying works from ReproZip bundles and why we posit that it’s great for archiving -- the many endless ways that unpackers can be added to replay the contents. It’s generalizable enough to use most any VM or container software; which means you can always get out and view what you pack in. We use Vagrant and Docker and Singularity right now, but if a new virtual environment or container software came out and gained popularity, an unpacker for that could also be easily added.

There are four unpackers currently available for your desktop, and one cloud unpacker as well -- ReproServer is our cloud-based application for re-running ReproZip bundles from the comfort of your browser alone.

The first unpacker for desktop is the directory unpacker (reprounzip directory) allows users to unpack the entire work (with all dependencies) in a single directory, and to replay it directly from that directory. This is unreliable if the application cannot be trusted, since it can point outside the unpacked directory. Hardcoded paths for example will still hit outside that directory.

The next is the chroot unpacker. Similar to reprounzip-directory, a directory is created from the bundle; however, a full system environment is also built, which can then be run with chroot(2), a Linux mechanism that changes the root directory / for the work to the directory. Although chroot offers pretty good isolation, it is not considered completely safe: malicious works might still escape to the host environment.

Third, the vagrant unpacker allows an experiment to be unpacked and reproduced using a virtual machine created through Vagrant VirtualBox. Using this, the work can be reproduced in any environment supported by this tool, i.e., Linux, macOS, and Windows.

Lastly, ReproUnzip can extract and reproduce experiments as Docker containers as well, which is another option for cross-platform unpacking and replaying. A Docker container, as opposed to a traditional virtual machine, does not require or include a separate operating system and is thus lighter and faster.

ReproZip also allows users to generate a provenance graph related to the execution of the work by reading the metadata available in the .rpz package. This graph shows the experiment runs as well as the files and other dependencies they access during execution; this is particularly useful to visualize and understand the dataflow of the experiment.

There is also a VisTrails plugin that creates a VisTrails workflow, that can be used to run the experiment in VisTrails.

In testament to ReproZip’s flexibility, we are also adding in an unpacker for Singularity soon!

19 of 23

Our prototype － extending ReproZip for news apps!

Data Analysis, App, etc.

reprozip

Executing

Tracing

ReproZip Bundle

.rpz file

Record

Add WARC

Preservation

Bundle

.rpz file w/ WARC

Unpack

reprounzip dj

Preservation

Bundle

.rpz file w/ WARC

playback

View news app locally, even without wifi!

Pack & record

Unpack & replay

20 of 23

Demo: Interactive Election Polls 2015 from

the Guardian

Source found at: https://github.com/guardian/interactive-election-polls-2015

This is an application from the Guardian that sought to predict voting patterns for the 2015 election in the UK. We used our prototype to create a completely archival bundle of the application!

First is packing the app: tracing + packing with reprozip. run reprozip trace and the application executes like normal during the trace so we make sure that everything is captured that lives on the server -- database, client-sever architecture, etc. then run reprozip pack.

Then we recording with the rz-dj extension, which captures files that don’t live on the server that we potentially need, like the css

You can see WARC data in reprozip package that is added after the recording step (second 42)

Then you turn off WiFi and replay the application from the RPZ file on a different system, and see it still works great!

21 of 23

Current Limitations of the Prototype

ReproZip only captures what was executed; some news apps require all parts of the environment (e.g. gems for Ruby) that may or may not be executed, so that all the links in the front-end are captured correctly�

Implemented extra rule to detect and automatically capture Ruby gems, will implement for other languages in next phase of dev work�

Access to external APIs and data (e.g. on S3 stores) when re-running the news app from RPZ file�
Tested on a few types of interactives of similar types (e.g. built on Rails and Node.js, Django) based on environmental scan of news apps �

More testing coming in the next phase of work and/or by anyone who wants to test out the prototype and give us feedback!

22 of 23

Next steps, funding to:

Continue generalizing the tool for a wider range of interactive sites
Build in functionality to point to archived, emulated web browsers
Create a GUI that works for newsrooms

23 of 23

Thank you! Questions?

Get this presentation: https://goo.gl/JKZKK5

Try out our prototype: https://github.com/reprozip-news-apps/reprozip-news-apps

Documentation: https://reprozip-news-app-archiving-tool.readthedocs.io/

Any feedback? Either leave us an issue on GitHub or email us: rz-dj@nyu.edu

We have lots of questions for you all, and would like as much feedback as possible from the data journalism community. What would work with your workflows? What archiving edge cases do you have? What software can we optimize for next? What types of software would your sys admins be willing/able to install on your production servers? This is ultimately a tool for you all to use, as again, in archiving the web via emulation, newsrooms would have to be participate in this process. Either you all or your sys admins would have to do the packing (libraries can verify fidelity and worry about the unpacking, making it discoverable to end users, making it secure for institutions displaying it to end users, etc) but newsrooms are the first step in the process. So ultimately this tool is only useful and successful if it works for you all!