CodeInk
Objective
Make it possible to think about and solve programming problems visually, in contrast with syntax-focused lines of code.
Focus on inherently visual problems, where the user’s first instinct is to pick up a pen and draw. Ex: list sorting or graph search problems from an algorithms class.
A typical algorithms class
Teachers explain concepts through concrete examples, by redrawing data structures over and over to show changes.
Students dive into problem sets after class with no way to test their conceptual understanding, other than writing code.
6.006: Insertion sort explained by redrawing the list. There’s some ambiguity between arrows representing the ‘key’ and arrows that indicate transitions between stages of the algorithm.
Proposed solution
A web-based visual programming environment. Every interaction on the stage results in a line of Python code.
Work in progress: people.csail.mit.edu/jscott/codeink
Very related work: Bret Victor on Drawing Dynamic Viz, Philip Guo’s Online Python Tutor
CodeInk’s Usage Model
CodeInk
Grasp a Problem
Animation
Make data visual
Manipulate data to describe algorithm
Steps are captured as you go
Code
Understanding
Making Data Visual
Drawing a concrete example on the stage
Built-in objects: numbers, strings and lists
The programmer can drag and drop these objects on the stage, and the system draws them using a standard visual vocabulary.
Custom objects:
The programmer establishes their own visual vocabulary by drawing and mapping visual variables to data attributes. The objects can then be dropped on the stage.
GraphNode
label : cost
fill : visited
arrows : edges
5
Direct Manipulation Programming
Manipulate the example to trace algorithm
Change data by transforming visual variables: color, labels, arrow endpoints
Use intuitive gestures to manipulate built-in objects like lists and strings
Insertion into a list
2
2
unvisited
visited
7
5
6
7
5
6
Steps captured as you go
System captures each action as a line of code. The stage reflects the example after the current step in the trace.
The steps can be scrubbed through, played back, and clicked to jump to a point in the trace.The trace can be shared, making CodeInk ideal for asking questions on a class forum, or collaboratively working on problem sets.
Research Directions / End Goal
The abstract, general algorithm
Will behave correctly on new examples
Tracing an algorithm through a concrete example
Correct behavior for *that* example
Replacing Powerpoint
Communication for teachers
Grounding understanding for students
Testing and debugging for devs
Novel programming environment
Describe algorithms visually
Quickly draw new examples and simulate behavior.
User’s description of algorithm
Use cases
for system
A sequence of changes to objects
The complete implementation
Including flow control, recursion, functions, auxiliary data structures.
System needs to support a visual and DM vocab for...
Current focus
Future stage
Ideas for next steps:
Describing Flow Control by Direct Manipulation
c
for each:
dot the objects you want to iterate over
codeink guesses at an iteration sequence
decisions:
show you want to branch by drawing...a branch�build an expression by dragging objects and writing
c
node.cost
Ideas for next steps:
Describing Flow Control by Direct Manipulation
Flow control context is visible on the stage
c
for every edge in node
step through the loop by dragging the slider
current state of loop is visible
Ideas for next steps:
Describing Flow Control by Direct Manipulation
Flow control context is visible on the stage
x
node.cost > 10
x
otherwise
swipe right
describe steps for first branch
describe steps for other branch
two sides of a decision
Ideas for next steps:
“Being the computer” in search/traversal tasks
Limit visible objects to what can be reasonably accessed in current scope.
Ex: for neighbor in node.neighbors:
node
Contrast: Programming by Example
Programming by Example involves describing outcomes for concrete scenarios, from which the system needs to generalize.
Possible CodeInk principle: enable the user to start with a concrete example, but move up the ladder of abstraction by describing flow control (just like an expert programmer) via direct manipulation.
In other words, let the user do the generalization, but make it easier for them to do so.