A Visual Functional Programming Language for Scientific Computing

Pete Vilter

CS Senior Thesis, University of Chicago

Advisor: Borja Sotomayor

Unofficial Advisors: Ravi Chugh, Mike Wilde

https://github.com/vilterp/thesis

TOC

• Why?
• What
• Demo
• Graph syntax / Semantics
• Codegen
• Progress Monitoring
• Future Work
• Swift
• Typechecking
• Files
• Architecture / Implementation

Why a new visual PL?

• Scientific workflows are functions:
• data in, data out; not specifying interactivity
• Relatively simple
• A graphical repr is good for:
• Easy editing (no syntax errors)
• Easy progress monitoring
• Existing solutions are limited
• List processing, subroutines awkward
• Contribution: visual syntax based on FP
• Non-goal: support super-complex code
• Textual code is more suitable past certain complexity threshold, because more dense

Stepping stone to:

• Compilation to Swift:
• distributed parallel execution on cluster
• Currently generates Python code which runs on local machine

Language Model

Editor

Language Model: Editing Python

Run it!

• And then aspects:
• Graph "syntax"
• Code generation
• Progress monitoring

Graph syntax

• Analogy:
• valid token sequence can be invalid syntax
• some edges in graph can be invalid
• No cycles
• No edges out of lambdas
• Function usages as value vs. application are mutually exclusive
• Design decision: function and application nodes are not separate

Code generation

• Generate single Python file from whole module
• Graphs: topological sort on application nodes
• Each node becomes a function call
• All functions take and return dictionaries of values

Generated code:

def main():

ap0_names = log_call(names, "ap0_names", {})

ap0_names_names = ap0_names["names"]

def lambda0(lambda0_ap1_add_excl_str):

lambda0_ap1_add_excl = log_call(add_excl, "ap1_add_excl", {"str":\� lambda0_ap1_add_excl_str})

lambda0_ap1_add_excl_added = lambda0_ap1_add_excl["added"]

lambda0_ap2_add_hi = log_call(add_hi, "ap2_add_hi", {"str": \

lambda0_ap1_add_excl_added})

lambda0_ap2_add_hi_added = lambda0_ap2_add_hi["added"]

return {"lambda0_ap2_add_hi_added": lambda0_ap2_add_hi_added}

ap3_map = log_call(Map, "ap3_map", {"function": lambda0, "list": ap0_names_names})

ap3_map_mappedList = ap3_map["mappedList"]

ap4_space_join = log_call(space_join, "ap4_space_join", {"list":\

ap3_map_mappedList})

ap4_space_join_joinedList = ap4_space_join["joinedList"]

return {"ap4_space_join_joinedList": ap4_space_join_joinedList}

Example: Word count

Spark:

​

text_file.flatMap(lambda line: line.split())

.map(lambda word: (word, 1))

.reduceByKey(lambda a, b: a+b)

Progress Monitoring

• Server sends updates on function call & return
• Client builds call tree������

​

main

…

Progress Monitoring: More

• Supports parallel execution (call tree w/ mult. stacks)
• Argument & return�values sent to editor�as soon as are known
• (can be inspected�at runtime)��
• Generated code:
• log_call(func, ap_site, args)

…

main

Progress Viz: Future Work

• Inspect nodes up call tree
• Progress bar for list HOFs
• Map, Filter, Fold
• Machine-level stats
• CPU
• Memory
• Disk
• open shell
• etc������

​

…

main

Future work: files

• Upload
• Insert as values onto canvas
• Note: functions cannot be run without all args present
• Use directories as lists
• Get files from network

Future work: Swift

• Wrap binaries
• Parallel execution
• Distributed execution
• Supercomputer and cloud platforms

Future work: typechecking

• Hindley-Milner for Graphs

Future work: typechecking

• Hindley-Milner for Graphs

Future work: typechecking

• Hindley-Milner for Graphs

Future work: typechecking

• Prevent invalid edges

Future work: typechecking

• Prevent invalid edges

Future work: typechecking

• Mark edges rendered invalid by external changes

ages

ages

Implementation: Basics

• Editor: 100% Elm
• Paradigm: Functional Reactive Programming
• Disciplined state management invaluable
• Typechecking invaluable
• Metrics
• ~1300 lines elm-diagrams library
• ~2700 lines project-specific
• + Elm stdlib (data structures, wrappers of browser APIs)
• Compiles to JavaScript
• 21k lines, much of which is Elm stdlib
• Backend: node.js server
• Serves code
• Executes python subprocess, streams results back to editor

Implementation: Time

• Vast majority: Graph Editor
• Why? No graphics lib with abstractions for
• relative layout
• absolute layout
• absolute layout derived from relative
• Made my own

Diagrams

← alignment →

Diagrams

Diagrams

• Functional combinators for relative & absolute positioning, inspired by Haskell lib (same name)
• Query display trees for:
• Coordinates of laid-out subtree
• What subtree mouse is over
• Rough graph editor code metrics

​