1 of 37

Programmer Experience When Using CRDTs to Build Collaborative Webapps:

Initial Insights�

Yicheng Zhang, Matthew Weidner, Heather Miller

Composable Systems Lab,

Carnegie Mellon University

2 of 37

Collaborative applications are like Google Docs

3 of 37

Multi-user applications are like Google Docs

4 of 37

Want to be local-first documents instead of

Google Docs

5 of 37

Need high availability: want to have fast updates

6 of 37

Need to have (eventual) consistency over Replicas

7 of 37

Need to deal with network partitions

8 of 37

Conflict-free Replicated Data Types (CRDTs) helps people to create multi-user apps

They even look like regular types

CRDTs have built-in replication & strong eventual consistency.

9 of 37

No known user study has been done to evaluate if CRDTs are helping to create multi-user apps

CRDTs are supposed to help people to make collaborative apps easier! Are they doing a great job?

10 of 37

Subjects are asked to build a collaborative animal shelter application

11 of 37

Task 1️⃣: a collaborative animal

​

​

12 of 37

Task 2️⃣: a collaborative cat or a collaborative dog?

13 of 37

Task 3️⃣: a list of animals that can be added or edited at different locations

One animal at a time,

need to flip page

14 of 37

We studied three libraries

Yjs Automerge Collabs

15 of 37

Yjs CRDT Library is widely used in doc editing applications

Yjs is used for Jupyter Notebook’s collaborative editing function.

​

16 of 37

Get a map, set the map. Done.

​

​

Yjs can be used just like regular maps and arrays

17 of 37

Get a map, set the map. Done.

​

​

Yjs can be used just like regular maps and arrays

18 of 37

Automerge CRDT Library is developed by Ink & Switch, who coined “local-first software”

19 of 37

Automerge CRDT Library is like changing a JSON file

Pass in a lambda function

20 of 37

Automerge CRDT Library is like changing a JSON file

The body here is just what people usually do in a local file

21 of 37

Automerge CRDT Library is like changing a JSON file

Send to peers and update DOM, do explicit message sends

​

(Automerge is not declarative)

22 of 37

Collabs supports users in creating their own data models (create new classes)

Collabs is a library developed by our research group.

On the other hand, the program will be longer

23 of 37

We recruited 15 subjects who know JavaScript to use one of those three libraries and watch how they code

24 of 37

Programmers ran into some issues while coding

25 of 37

{“cat” : {“name” : “ABC”, “Height” : 10}}

Initializing: {“cat” : {}}

{“cat” : {}}

{“cat” : {}}

Replica 1

Replica 2 joined later

Remote update

26 of 37

Should check if there is other preexisting replicas

{“cat” : {“name” : “ABC”, “Height” : 10}}

{}

{“cat” : {“name” : “ABC”, “Height” : 10}}

{“cat” : {“name” : “ABC”, “Height” : 10}}

Replica 1

Replica 2 joined later

Remote update

Check

27 of 37

Participants have poor understanding of Replicas

28 of 37

Issue 2: confusion between local (non-collaborative) and remote (collaborative) data structures

​

Update DOM when CRDTs change

Collaborative

29 of 37

Issue 2: confusion between local (non-collaborative) and remote (collaborative) data structures

​

One animal at a time,

need to flip page - local

30 of 37

Issue 2: confusion between local (non-collaborative) and remote (collaborative) data structures

​

Forget to update DOM when “page” change by calling refreshDisplay()

31 of 37

Issue 2: confusion between local (non-collaborative) and remote (collaborative) data structures

​

​

  • Can usually realize the problem without hints from researcher might because they wrote the data structure by themselves.

​

  • They might not have any more errors later.

32 of 37

Issue 3: All libraries studied presented both strengths and weaknesses

Library

Pros✅

Cons❌

Automerge

✅JSON is familiar for JavaScript Developers

❌Double initialization problem

Yjs

✅Easy to use maps and arrays

❌Difficult for large, long-lived applications

Collabs

✅Support creating classes

❌More up-front cost for users to learn

33 of 37

Next step: do the qualitative coding over the User Test results

34 of 37

Further in the future: fix those problems (APIs/tools/etc.) to make CRDTs easier to program

35 of 37

What we would like to get out of this workshop:

​

​

​

  • Suggestions for how to do rest of study? e.g., coding

​

  • Any issues with the study design?

​

  • How to make CRDTs libraries easier to use?
    • how do we iterate on these results?
    • how do we test improvements?
    • do folks see obvious ways to improve these libraries?

36 of 37

37 of 37

The double initialization problem

​

​

​

  • Particularly problematic for Automerge. Can occur both at initialization and for subsequent updates.
  • Less problematic for Yjs and Collabs ostensibly due to declarative interface? If it happens, it tends happen only during initialization. Can’t happen for subsequent updates

Backup slide: