1 of 21

Code As An Artistic Medium

Bomani Oseni McClendon • @bxmani • bomani.xyz

2 of 21

3 of 21

Code is a medium!

Is code is a medium?

My belief that code is a medium has inspired me to explore creating an art practice through code. It’s something I’ve mentioned a lot in speeches and conversation,
But earlier this year I began to think about the topic more deeply after an experienced artists challenged the concept.
In 2019, it’s clearer than ever that code can be used to create a wide variety of art
Many people here in the room are great examples. You’re using code as a foundation in your art practice.
There are now also a variety of frameworks that allow us create art with code more easily. Tools like Processing, P5, OpenFrameworks, and others have made it much easier to build an artistic practice that is based on thoughtful use of code as a medium.
But as I thought about this topic, I also wanted to compare code to other artistic mediums. Specifically, I focused on analyzing the differences between code and poetry.
Similar to code, poetry is often expressed through text.
But interestingly, poetry can express meaning beyond its literal interpretation. poems express deep multi-faceted meanings using words, rhythm, and other features and techniques of language.

4 of 21

“For My People” by Margaret A. Walker

“For my people everywhere singing their slave songs repeatedly: their dirges and their ditties and their blues and jubilees, praying their prayers nightly to an unknown god, bending their knees humbly to an unseen power;”
“For my people lending their strength to the years, to the gone years and the now years and the maybe years, washing ironing cooking scrubbing sewing mending hoeing plowing digging planting pruning patching dragging along never gaining never reaping never knowing and never understanding;”
In this poem, the verb repetition reinforces the feeling of an exhaustive list.
The the formatting helps to visually separate each topic, and tabbing here highlights the subject of each section. This provides a separation and hierarchy that may not have been interpreted if the text wasn’t in this format.
In summary, this poem uses textual language as a medium. It both conveys the overall idea of the piece through the literal interpretation of the text – but it also uses other techniques to express or reinforce additional ideas.
Code, as a medium, arguably doesn’t provide artists with the same affordances. Largely, the value of the code in a creative project is only defined by its utility; by its ability to make something work.

5 of 21

6 of 21

7 of 21

Is the value of the code as a medium only defined by its utility, or can it have value beyond that?

8 of 21

“prog.c” by endoh2 for IOCCC

9 of 21

“Shape of Code” by Joseph Wilk

10 of 21

“CodeProfiles” by W. Bradford Paley

11 of 21

From Code to Visual

Idea Introduction

But to help build a deeper affinity with the topic, I decided to prototype my own translation from code into visual
I’ll explain my approach to help give you a sense of how one might go about approaching this problem.
My goal was to directly translate code into visual art.
I asked: What if each piece of code could have its own unique visual ‘signature’, which changed based the way the code was written.
Rather than just focusing on the functionality of the code, the actual text that comprises code is used to produce visualizations that are unique to the structure/implementation of the code.
In this model, two pieces of code that do the same thing will produce different visuals based on the variable names, tabbing, line breaks, etc they include.
And furthermore, this model could create a visual representation for any piece of code, regardless of what it does. This means, for example, that even, server-side or ‘backend’ code that doesn’t functionally produce any visual output could still also have an artistic visual piece associated with it.

12 of 21

13 of 21

Works from “Komputerstrukturen” by Peter Struyken

14 of 21

ASCII Table

15 of 21

hello

104 - 101 - 108 - 108 - 111

16 of 21

Here’s a picture of the first visual I made as a result of a direct translation from code, which was produced by writing a simple program that would place rectangles in a visual according to this rule.
After evaluating this first approach, I decided that I wanted to test making these visual representations dynamically, so that the visual piece could develop concurrently with the development of the code.
To prototype this dynamic behavior, I created a plugin for the code editor Atom (that I was using at the time) that could send the in-progress code directly from the code editor to a web application I built that creates visualizations of the code using p5.js.
For this iteration, I also added color. Each character is represented by a semi-transparent rectangle that is color based on its syntax highlight - the coloring the the editor adds to parts of the code to help make it more readable.

17 of 21

18 of 21

I started to have favorite files, based on how much I liked the visual representation that they produce. Here’s a reel of a few visuals that I enjoyed.
But I also started to write code different to help shape the aesthetic output. For example, changing the length of my line, adding new lines in unique places, and so on.

AST Ideas

Much of the work that I did to explore this topic was done late last year, but I am interested in exploring to topic further.
So I want to share one idea for a future direction this topic could be taken.
The prototype I shared, and many of the others I’ve seen, focus on using the characters that make up the code the primary input to the visualization.
This approach is useful and generalizable; it can be used on any file of text. But this approach doesn’t include a true interpretation of the code. It treats the code as a bundle of text, and it doesn’t leverage the unique aspects that differentiate code from other forms of text.
Instead, what if the meaning of the code could also be represented in it’s visual translation?

19 of 21

Abstract Syntax Trees (ASTs):

Abstract representations of the syntactical structure of code.

To approximate this, I’ve considered the use of Abstract Syntax Trees, which I’ll explain briefly
Abstract Syntax Trees, or ASTs for short, are abstracted representations of the syntactical structure of code.
We call it abstract because the AST doesn’t aim to represent every detail of the code (like semicolons or spaces), but instead focuses on the structural content.
ASTs are often used to help verify the semantic correctness of code OR to create a generalized representation of code that can be generated into programs written in variety of languages.

For example, you could take a piece of Java code, generate an AST from it, and then have it generated as functional Javascript code

But I wonder what it might look like to generate AST an from code, and then use the AST to produce a visual.
Using this approach, the visual could be produced based on a deeper understanding of the code itself. For example, we could color visual elements representing the conditions of an if statement red and conditions of a while loop green.
While I haven’t yet had the chance to experiment with using ASTs as part of the translation process from code to visual, it’s something I’d love to try out. If you’re interested in exploring this space with me, please get in touch with me after the talk :)

20 of 21

“It is thus through the work of artists that the artistic possibilities of an artistic medium can be discovered, and not by analyzing the material in isolation.”

Internet Encyclopedia of Philosophy

21 of 21

Thanks

Bomani Oseni McClendon • @bxmani • bomani.xyz