1 of 21

Pickling for Clipboard API

huangdarwin@chromium.org

Darwin Huang (he/him)

​

2 of 21

Agenda

  • Current Spec Status
  • Problems
  • Pickling design
  • Questions?

3 of 21

Current Spec Status

  • DataTransfer
    • Synchronous. Support for text, html, and images.
    • Blink and Webkit both implement an unstandardized and minimally documented custom format implementation (based on Pickling)...
  • Clipboard API:
    • Asynchronous, promise-based API
    • Early 2019: Specified navigator.clipboard.{read,write}()
      • new ClipboardItem object instead of the previous synchronous DataTransfer concept.
    • Blink and Webkit both implement support for text, html, and images.
    • No support for custom formats

4 of 21

Problems - niche formats

CAD design web apps (SketchUp)

Niche, custom formats (CAD formats)

Image editor web apps (Photopea)

Niche, custom formats (images with layers)

5 of 21

Problems - proprietary formats

Document editors (Google Docs)

Proprietary formats (document formats)

​

6 of 21

Problems - metadata

UX design web apps (Figma)

Formats untouched by sanitization (Images with all metadata preserved )

7 of 21

Problems - interop with native

Figma (or other sites)

...with native app counterparts

Native apps interested in interop (Figma)

Interop with custom formats

8 of 21

Problems - interop with native

Office Online (or other sites)

...with native app counterparts

Native apps interested in interop (Microsoft Word)

Interop with custom formats

9 of 21

Pickling Design

  • Provide web applications with access to custom formats.
    • Web-originated, so no interop (or RCE concerns) with legacy native apps.
    • Unsanitized, because formats are custom.
    • Native apps may opt in to using custom formats.
  • Still recommend existing sanitized APIs for general use, for wide compatibility and safety.
  • Context: Last year, Pickling was suggested when discussing raw clipboard.

10 of 21

Existing Write

11 of 21

Pickling Write

12 of 21

Existing Read

13 of 21

Pickling Read

14 of 21

Basic Protections

  • Secure Context
  • Active Frame
  • Permission
  • Consistent with many other new, “powerful” APIs
  • User gesture, and optionally feature policy (recently renamed to permissions policy).

15 of 21

Proposed mapping

  • Open to alternative mappings.
  • Example: “custom/foo-format”
    • Windows:
      • Capital words separated by spaces.
      • Web Custom Foo-format
    • MacOS:
      • UTI reverse-DNS naming
      • com.web.custom.foo-format.
    • All other platforms (Linux, Android, ChromeOS, etc):
      • MIME types
      • application/web;type=”custom/foo-format”

16 of 21

Why map formats to native

  • Sites should consistently use MIME types across platforms, to avoid platform-dependent code.
  • Browsers should be able to interop with one another.
  • Use consistent native formats between browsers.
  • Different per platform, to follow platform conventions.
  • Native applications may also opt in by implementing support for these formats.

17 of 21

Privacy Concerns (navigator.clipboard.read)

  • Some sites may provide excess metadata or PII on the clipboard. This will be less transparent with pickled formats, which aren’t pasted everywhere.
  • Ex. Some image formats provide metadata with GPS and camera information.

18 of 21

Security Concerns (navigator.clipboard.write)

  • Previous related raw clipboard proposal’s arbitrary code execution concern alleviated.
  • Some sites may place scripts on the clipboard. This will be less transparent with pickled formats, which aren’t pasted everywhere.
  • Ex. Pickled SVG or HTML might contain scripts.

19 of 21

Other

  • Sorry, no public signals yet...

20 of 21

Questions?

  • Open-ended overall, but please separate some concerns, such as read/privacy and write/security, into separate topics.
  • Please also provide feedback on API shape.

21 of 21

After the presentation...