A small tool for �sketching UIs

• Embeddable
• Simple collaboration model
• Acts as you intend – no manual stack management
• See also: Similar tools and open questions

A small tool for sketching UIs

Have an editor to quickly create simple UI-sketches, �usable e.g. for discussing how an interface should look like.

Jan writes:

how about we try this:

[Tool created sketch of UI]

Anna writes:

I think a vertical layout is better, like so:

[Tool created sketch of UI]

add comment

A small tool for sketching UIs

Rough sketch of what it might look like, showing the three tools/modes.

​

Select

New UI element

New annotation

A small tool for sketching UIs

It should be simple and compact to be embeddable in other tools (e.g. fossil, gitea, mattermost etc.)

​

Add a UI sketch!

Metaphor

• If figma is for creating a blueprint of your UI, using a pens, stencils and squared paper
• …then this is drawing with a sharpie on a sticky note

​

• Collaborating in figma is like refining the blueprint,
• Collaboration here is responding with a sticky-note sketch of our own, building upon their work.

RELEVANT FEATURES

Autogrouping: No grouping or stack management

• When you drop a UI element on another UI element they become grouped; inner elements are always on top of outer ones; if you drag the outer element, all inner elements are dragged along.
• This means: No stack management with „send back“, „send forward“ or „group“.

​

This actually works and has been done before:

• Try at https://jdittrich.github.io/quickMockup/�or https://whimsical.com
• Description at: https://www.fordes.de/posts/autostackautogroupinterfaces.html

Zoom to cursor, panning etc.

• Basic interactions like zooming and panning should work smoothly – zoom needs to move the canvas in way that the same canvas-point stays under the cursor. Some apps don’t do that, which feels confusing.

Contextual UI

• The UI should show contextually relevant actions at the elements that can be edited.
• Usability test will be needed to find most important features and commands.
• UI should be visible, it does not fake minimalism by hiding

�Prior art: A lot, but most closely to this idea: Whimsical uses such a contextual UI.

COLLABORATION MODEL

Collaboration model

• The collaboration model assumes online collaboration in a thread- or stream-like structure, e.g. in issue trackers, discourse discussions, chats…
• UI Sketches could be
• created from scratch
• copied from a previous post (“How about changing…”)

Collaboration model

• It is easy to build upon one’s own or other user’s sketches: Just create a copy and make your changes
• Building upon each other’s work is thus like responding in a conversation: you said… and I would add…

build upon this sketch

Collaboration model

Using a copy-and-build-upon model of collaboration, we gain the following advantages: �

• This way, no hard-to-implement conflict resolvement is needed (E.g. Conflict-Free Replicated Data Types, Operational Transforms, or less complex, statemachine-replication).
• History is easy to understand (the things you copied from are still there)
• Combination with text (e.g. in comments or issues) is easy as sketches mostly behave like images or text once posted (editable by the author if needed, but mostly static)

ALTERNATIVES

Whimsical

• Contextual UI
• Autogrouping

• Not open
• Many different usecases
• Live-collaboration tool
• Not sketch-like looking

Balsamiq

• Looks like a sketch (yey!)
• Focussed on UIs
• Similar target group – people collaborating around designs

• Some functionality is clunky (no zoom to cursor)
• No auto-grouping
• Still too many features
• not open

Draw.io

• Open
• Established codebase
• embedded in some products via plugins already

• Universal diagramming tool, no focus on UI
• No auto-grouping

​

Excalidraw

• General whiteboarding
• No auto-grouping
• I still need to figure out how their code works (some combination of react+canvas; node 18-20, yarn required to run)

• Looks like a sketch
• Open

​

IMPLEMENTATION: DETAILS FOR DEVS

More backend-y stuff I tried or thought about. It is written in the context of drawing apps, so I guess, just like understanding a web app usually needs knowledge of concepts like routers, requests, etc. it might be hard to grasp without knowledge of drawing apps.

Repository

It does only a part of it yet, but if you want to try:

https://github.com/jdittrich/quickwire

Expect it to barely look better than this!

OOP

• Most tools I looked at seem to to avoid an MVC-like approach, instead they have figure objects with (sometimes rather elaborate) inheritance which combine model and view. Controller is separate (tool, command)

Hotdraw, morphic and OOP

• OOP seems to work well for this kind of applications

�Classic Examples:

• hotdraw
• A diagramming framework, originally implemented in smalltalk
• There are some papers and discussions of architecture on hotdraw, making it relatively easy to understand
• morphic
• more focussed on the creation of UIs, also originally created in smalltalk
• Some publications, but less than for hotdraw, afaic

Some approaches not used so far

• Event-stream functional
• I found it hard to wrap my head around and I did not find a lot of prior art.
• vue or react-based
• I know that some projects do this (e.g. excalidraw is based on react), so it can work
• I had the impression that what these frameworks were good at and what interactive drawings required was not aligned
• d3
• it seems to be more aligned than vue/react
• its still not a data visualization and I found no prior art
• Zoom and drag/drop seems to work well, though

Canvas and figures

• The sketch is displayed on a canvas, which can hold content: Figures
• The canvas can be zoomed and panned (coordinates are translated from screen to document space)
• Figures have…
• a position, width, height
• A parent (another figure or the canvas)
• Attributes like a text, if it should look active or passive etc. (these depend on the “subtype” of figure (Button, Textfield, Box, etc.; This can be implemented as an OOP subtype, but there are also other ways)

Layers of rendered elements

(Entirely internal, this is nothing the user deals with)

typical UI elements like „duplicate this element“-button or „select size“

handles: scaling, changing size etc.: little boxes to drag/drop

element previews

document itself (changed by commands)

Notes on language and tools

• So far I used toolchain-free light setup:
• Javascript Typescript with ES6 Modules and erasable types.
• Unit tests via ES6 module imports and qunit-in-the-browser

​

• Advantages are:
• Easy to pick up development, only a simple http server is needed (e.g. the one coming with python) and Typescrit
• Easy to debug in the browser, there are no sourcemaps or long call-trees.

Selection

• The selection can be empty or have one or more figures
• Most commands done would affect any object in the selection
• Thus, instead of saying „use this command on these objects“ we will usually say „use this command on the selection“
• Not sure if only selecting one element is sufficient, autogrouping might do away with much of the need to select several elements manually.

Tools

• Tools determine what a certain input does.
• …however, they are not the same as selectable tools in a sidebar
• e.g. Pressing the mouse button on a handle of a selected figure activates the „resize“ tool; then, moving and releasing the mouse would commit the „resize“-command.

• Tools have methods like „onmousedown“, „onmousemove“, „onkeypress“ etc.
• The methods are called by a the toolManager that knows the active tool and can transition between them
• OOP-pattern: State

​

Commands

• Commands actually change the document
• Commands are usually issued by tools
• Commands are undoable
• Like with the tools, there is a commandManager to call methods like do(new CommandType), undo() and redo(),

Theory: Why such a tool?

• Why open source designers need tools beyond text and code

Examples for sketches

Some examples for the level of fidelity that the tool should be able to realize

By Members of WMF Growth team: Wireframes from WMF Growth team meetings, CC BY-SA 4.0, Source: https://commons.wikimedia.org/w/index.php?curid=79612907

Wireframe brainstorms by sonali sridhar, CC BY-SA 2.0, Source

​

Paper draft - wireframes by Emma Jane, CC BY 2.0, source

​

​

Wireframes by Christine Ronde , CC BY 2.0, Source

https://mastodon.blaede.family/@cassidy/116404709985792379 “I found some exploratory sketches from 2013 (I would often doodle things out when talking them over remotely, and then let we’d iterate and then someone better than me with design tools would make a pretty pixel-perfect mockup)”

Most on Wireframe Sketches Gallery

Stickyframing

Local Leaderboard

Website

Paper Wireframe

Paper Wireframe 2

Notebook

Archive: Old open questions

Open questions: Rendering

• Canvas it is!
• It seems the easiest to just render to HTML/CSS: All elements are boxes anyway.
• Some tools render to SVG (Penpot, Whimsical) other to canvas (hotdraw, morphic)
• For ease of sharing/displaying-as-image, rendering to a canvas/a png would probably make sense.
• Details:
• On canvas, the sketch-like look probably needs to use rough.js
• I don’t know how to handle text on canvas
• most implementations on canvas use a quad-tree to optimize rendering via „dirty“ flags and redrawing parts.

Former Open questions: Rendering 2

• A quirk of rendering to HTML that I noted is that, whenever the stack order is changed (internal “to front” or “to back”, not called by the user), I also need to reflect this to the HTML by calling something like private toFrontHtmlRenderer().I guess this could later be solved by the view listening to events and then calling its own needed methods. Rendering on canvas would just redraw the “dirty” areas with the correct stacking (I guess there the complexity is in when areas are marked “dirty”)
• I went with canvas rendering. I do full redraws, so dirty-marks are not a thing at the moment.

Former Open questions: Overlays, previews

• A drag or resize is „done“ when releasing the mouse-button, but needs to show feedback while dragging. So:
• should I actually change the figure while dragging and, if cancelled, reset to a stashed initial state OR
• Should only a preview overlay be dragged and the figures are changed when the command is committed. OR
• Use Model/View-Model? (change only view-model, then commit to model)
• Changing a preview

Open questions: Overlays, previews

…via Transient mode: During drag, changes are transient

​

onDragStart(){

state.transientMode(true) …

onDragMove(){� selection.move(…�onDragEnd(){

state.transientMode(false)

commandManager.apply(…

…probably via Memento-Pattern on the complete state?

Former Open questions: Overlays, previews

Strategy: Dragging a copy, updating model on drop.

On Dragstart, make the original view invisible; create a copy that is rendered on top. Drag this copy. When dragging is done, destroy the copy and change the position of the original.��(Works pretty well in MVC, had problems in declarative-reactive paradigms)

​

Worked great!

Fromer Open questions: Abstracting auto grouping

Most models of graphics work based positions, manual grouping and stack management. Here, we would only have the user-issued position. Should figuring out the grouping happen: �

– before I construct commands (so the commands themselves operate on the stack-grouping model; more deterministic)

– after I issue the commands (so the commands just contain the positions, the grouping is part of the drawing-object behavior; more reflecting user action)�

…I tend to choose the latter one (but I will probably need escape hatches then). It turned out the “before” was easier to implement (hit test on drop)

Former Open questions: Storage format

• Due to the nesting, XML might actually make sense…
• …but JSON is fine, too.

​

• Its good if it is human-readable, however, it is not primarily a plain-text-format or even a UI specification language
• Went with JSON for now