1 of 40

New Ways to do Peer Review

and Distributed Science

Michael Toomim, Katherine Baney, Raphael Walker, Matt Akamatsu, and friends.

Invisible College at Astera 2022

2 of 40

3 of 40

The game is getting in the way of Science!

How Invisibles change Academia

4 of 40

Two Innovations in Science Dialogue

  1. Subjective Peer Review
  2. Web of Thought

5 of 40

Subjective Peer Review

6 of 40

Peer Review

7 of 40

Peer Review

  • Share results
  • Filter attention
  • Get collaborators
  • Get users
  • Central control by authority
  • Hidden process
  • Majority rule stops paradigm shifts
  • Small sample of reviewers
  • Slow

8 of 40

Subjective Peer Review

9 of 40

Subjective Peer Review

  • Web of trust
  • No corruptible authority
  • User controlled
  • Process is open

  • See audience reaction
  • Connect and build community with shared values
  • Fast turnaround of feedback
  • Get collaborators

10 of 40

Demo: https://braid.news/

11 of 40

Braid of Feeds

12 of 40

13 of 40

14 of 40

15 of 40

16 of 40

Nuanced voting with Tags

17 of 40

Nuanced Voting with Tags

18 of 40

Voting with a Tag contributes to that Tag's Feed

19 of 40

Virtual Voters

Tags aggregate into Virtual Voters

"What does Astera think about this?"

"What does UC Berkeley think?"

"What does California think?

"What does Monsanto think?"

"What do funny people think?"

20 of 40

Anti-Gaming Incentives

Scenario 1: user trying to maximize influence

Scenario 2: east vs. west coast CRISPR tribes

21 of 40

Web of Thought

22 of 40

NOT SPECIFIC!

  • High stakes
  • No incentive to share negative results, partial results, experience of scientist, or data that others can build on
  • Rigorous expert feedback
  • Archival

23 of 40

SPECIFIC!

24 of 40

<point>

25 of 40

26 of 40

27 of 40

28 of 40

29 of 40

We need developers and users:

toomim@gmail.com�

Open discussion after this meeting:

https://braid.org/zoom

30 of 40

Legacy Peer Review Subjective Peer Review

  • Goal of reviewer: help the system
  • Reward: ??
  • Goal of reviewer: improve my feed
  • Reward: reputation

31 of 40

32 of 40

Outline

IC Orientation

Problem: We need new Peer Review and Publishing things!

  • Subjective Peer Review
  • Web of Thought

Subjective Peer Review

  • Problem
  • Solution
  • Demos

Web of Thought

  • Problem
  • Solution
  • Demos

Call to action

33 of 40

34 of 40

The game is getting in the way of Science!

35 of 40

Robert Boyle & friends left Academia in the 1600s

36 of 40

Gathered outside institutional walls in the “Invisible College”

Robert Boyle & friends left Academia in the 1600s

37 of 40

Gathered outside institutional walls in the “Invisible College”

Became the Royal Society, and invented peer review while deciding which papers to share in their journal

Robert Boyle & friends left Academia in the 1600s

38 of 40

Gathered outside institutional walls in the “Invisible College”

Peer Review took over Academia!

Became the Royal Society, and invented Peer Review while deciding which papers to share in their journal

Robert Boyle & friends left Academia in the 1600s

39 of 40

Gathered outside institutional walls in the “Invisible College”

Peer Review took over Academia!

Became the Royal Society, and invented Peer Review while deciding which papers to share in their journal

Robert Boyle & friends left Academia in the 1600s

The game is getting in the way of Science!

40 of 40