Published using Google Docs
hello from the 90s
Updated automatically every 5 minutes

Hello from the 90s.

I’m Howard Butler.

I don't know very much JavaScript,

and I think I'm here because I'm good

at tweeting pithy in-jokes for

geospatial programmers who came up in the late 90s.

We all know how what can happen

when people are chosen for

their tweeting abilities.

I hope to provide some nuggets about

platforms for geospatial software,

overlay my software development history with GISes

describe how platforms are mixing

together faster than the 90s and 2000s

describe how good open source

software **build with** rather than **instead of**

show a little LiDAR in a browser,

and talk about how this all relates to our now favorite computing platform

-- or at least the one for this conference -- JavaScript

** colosseum slide ** 

The foundation of one software civilization is the bones of another.

Like many, I got to geospatial software development in an unplanned and happenstance way.

** apple computer slide ** 

While I had a 80s nerd starter kit of an Apple IIgs

with BASIC and a 40 mb hard drive,

the reality that I would become someone

who makes their living giving away software

is still astonishing to me.

** tommy slide ** 

I grew up on a corn and soybean farm in Southern Minnesota.

I spent many summer days and fall nights in the fields,

Helping to plant, maintain, or harvest crops.

Throughout high school, I was given more responsibility and information about how to successfully do that.

Scratching out a living in the dirt wasn’t just a metaphor,

and I went to university looking to become better at it.

** agronomy slide ** 

I bounced around a few majors in college,

including some computer engineering,

before I settled on agricultural technology development.

I even took a course series

at Iowa State University conveniently nicknamed


Super Dirt,

and Super Duper Dirt.

Getting closer to graduation,

I started looking into geospatial technologies

to make a difference

for crop farming in the upper Midwest.

I soon came across a software program that would make that all crystal clear for me

** arcview slide ** 

ArcView 3

ArcView gave me the ability to overlay whatever I wanted.

** population map slide **

In the hands of a newbie,

it's like an automatic spurious relationship finder, and

** population map slide 2 **

I was fantastic at creating horrific cartography

and computing indescribable statistics

of mostly just population maps.

** population map slide 3**

Eventually, button and menu functionality weren't enough for

me or my major professor,

and I needed to script to graduate.

** avenue cover slide**

Avenue my first real workaday computing platform

Avenue was a SmallTalk-related language

that powered the GUI and data

processing objects in ArcView 3.

Before I started with ArcView, I had the basics of software development down --




but in the course of three or four years,

I became a max-level Genji with Avenue.

** genji slide **

With it, I could DPS my way through anything in ArcView.

I could sling vectors and rasters around with impunity,

but despite my skills, I was pretty easy to one-shot.

Avenue was a closed system with a weak software ecosystem around it.

There was some user-to-user script sharing,

and it did allow scripts to call

shared DLL libraries.

At the 1999 ESRI users conference, I saw

power utilities mapping electrical grid

energy state in real time button clicks.

God, I know this sounds lame now,

but in the time of cargo pants, it really wasn't.

** rootbound slide **

ESRI became rootbound by the success of ArcView, however.

Many users despised Avenue for some of the same reasons people dislike JavaScript.

ArcView also integrated poorly with COM for Windows connectivity. If you think cargo shorts are ugly, write some COM software

** netscape slide **

ESRI pulled a Netscape and rewrote

themselves from the ground up to be

Windows First and

Windows Everywhere.

I’m sure they had their reasons.

** bubba slide **

Along the way they jettisoned

my GIS security blanket -- Avenue.

** Dodson slide **

During the transition, an ESRI Canada programmer

named Bruce Dodson

posted a DLL that hooked Avenue

to the Python programming language

and allowed you to eval whatever

you wanted through either interpreter.

Python 2.0 had just come out, and geospatial software in the Python space was dominated by GDAL and PROJ Python APIs

Ok, maybe dominated is too strong of a word.

There were GDAL and PROJ Python APIs,

but that was about it for geospatial Python at the time.

Yes I know those Python APIs still look

like they did 18 years ago.

** AVPython article slide **

Python access in ArcView gave me a transition path

that allowed me to build upon my past work

plus it gave me a massive productivity boost

in a batteries-included language with an open source ethos.

** avTerra slide **

I jumped into Python and wrote an

ArcView extension called avTerra that

downloaded TMS map tiles into ArcView map views --

except instead of TMS


** TerraServer slide **

it was called Microsoft TerraServer

and there was only one server on the whole internet.

Instead of any sane URLs and JSON,

it used a cargo shorts kinda thing called SOAP to do its communication with XML.

Python gave me super powers, and

I was able to use it to fill map views

with ten-year-old 3 meter black and white imagery.

I still take solace in knowing how many grad students

I freed from the drudgery of digitizing from paper maps

to enable heads' up digitizing in ArcView.

Oh, yeah, JavaScript.

** Java logo/shorts slide **

Actually, let's chat about Java for a minute.

The cargo shorts developer crowd

was quite excited about the grand

promise of Java in the 90s

-- write once and run everywhere --

but the devils' bargain to be struck was rewriting

your software to live on the Java Virtual Machine.

Java succeeded poorly for many, especially in the 90s.

Early on, there was a speed disparity

between native code and Java.

That’s been gone for a long time,

but the impression stuck.

** GUI programming slide *

GUI programming in Java was the normal amount of GUI-programming-is-hell

plus dealing with Java's Swing API.

JNI, or Java Native Interface,

would seem to be the integration path,

but 20 years of history shows it to be

for Himalayan mountains goats

rather than a 6 -lane California highway of integration bliss.

The dominant problem of the time was

native windows applications,

and 90s Java was poorly suited for it.

Java didn't exactly fail, it's still everywhere, even today.

The market transition to Java didn't go as intended, however.

It didn't end up replacing everything and running everywhere.

The reason why was the cost of the bargain was too high --

porting working code to new environments

creates new bugs in addition

to carefully copying over your old ones.

The bargain meant tossing out battle tested designs.

Java poorly allowed systems to build upon their past,

and the entire global software library had to be rebuilt from scratch to live in Java.

Decades long investment still hasn't produced

a feature-parity pure Java GDAL or PROJ library.

Instead of one-for-one replacing of existing systems,

Java attracted new development that desired the JVM,

and older systems stayed where they were --

Ultimately tied by some C or Fortran compiler on native platforms.

In contrast to Java, Python

focused on integration with native code.

That investment has paid off in many of the

numerical computing, machine learning, and geospatial codebases  now using it as their primary language.

Ah yes, JavaScript ...

Just a second, let's talk some more about platforms.

In ten years, in 2027, what platforms will still be alive?

** windows slide *

Windows should still be here.

Microsoft proved that getting their heads bashed

in on mobile can do wonders for their ability

to think about software development platforms.

They now make tools that play well with others.

They build on and with things they dont actually make or control.

They actually now care that you're using their tools.

The Microsoft in the time of peak cargo shorts only sweatily claimed they did all this.

** android slide *

Android will still be there.

Android is our friend Java and a bunch of JNI.

It's even GUI programming in Java too.

Android is a platform with two billion users, with the most entrenchment on the most successful hardware ecosystem in the history of computing.

** iOS slide *

iOS of course won't be dead by then.

iOS has the snobbiest installed base,

with its jail-like pastiche, emphasis on security, and stockholm syndrome, users can be counted on to ask for another.

** OSX slide *

OSX will be dead by then.

OSX is the Lannisters of platforms.

It needs to pay off its technical debt at the Iron Bank of Braavos, and It looks pretty but kills everything it touches.

** Linux slide *

Linux of course.

As a containerized platform, or a platform of containers.

Either way, Linux is the “78 percent of air is nitrogen” of platforms -- it is so assumed we don't even bother to think about it.

** Browser logos slide *

Finally, the one we care about, JavaScript.

Every browser has a JavaScript engine, and

just about every computer has a browser.

JavaScript is a platform that encompases

every one of the native platforms

I just talked about.

This platform is as strong as ever as evidenced by

this conference and its evolution from talking about

novelty browser mapping capabilities

to full scale replacements for the entire spectrum of GIS.

I hate JavaScript --

Ok, maybe that's rather cliche.

Maybe you all hate JavaScript too.

I don't hate JavaScript because

the language is awful and arbitrary.

I don't hate JavaScript because of

JavaScript Framework Bingo.

I don't hate JavaScript because

I think significant whitespace is a

perfectly reasonable way to block code.

I hate JavaScript because the platform

once forced me into Java's devils' bargain

of rewriting everything I had already written.

A thirty year old, battle tested cartographic projection library?

A geometry algebra engine?

A geospatial format translator?

A lidar compression library?

We want all of these things in JavaScript

the same way we have them in many

other languages, but ports are worse than forks.

All of the platforms I described earlier aren't equal,

but they all have something that can target them

** LLVM slide *

LLVM is just a name now,

although low-level virtual machine might once

have been a fitting acronym.

It has quietly infiltrated most

developer tools in recent years.

Native mobile developer tooling and continual investment

in LLVM by Apple and others pushed it over the threshold of resistance futility.

LLVM is compiler technology.

It's different than others in that it breaks the

compilation problem up into pieces.

The typical compiler scenario we might think of

is having a C or C++ frontend and

a machine backend that is x86 assembly.

Or how about a Python frontend and a JVM backend?

Or, even better for us, a C++ frontend and a JavaScript backend!

This exists today.

You can use the LLVM toolchain to compile C or C++

into LLVM intermediate representation, or IR, and target JavaScript as a backend with Emscripten.


The tools to google are asm.js, WebAssembly, and Emscripten.

These tools are in use today.

My company, Hobu, Inc used them to compile the C++ LASzip lidar compression library into a JavaScript module.

It only performs a few multiples slower in a browser than it does on bare metal.

Every release tick of the browser and every release tick of the compiler stack has made that multiple decrease.

** plasio slide *

Many of the browser-based point cloud visualization libraries use it, including Potree and Plasio.

It's not a panacea.

Compiling the library to JavaScript actually forced us to completely refactor the LASzip library.

It was probably equivalent to porting it to a new


** denmark movie slide *

At the time, Emscripten implemented every c++ virtual table dereference as a function call in JavaScript.

This meant a simple C++ dereference of a polymorphic variable was three or four function calls.

It took a month to refactor it, and along the way,

we added open source niceties like --- tests.

If we had simply ported it to JavaScript,

we'd have burned a month forking

the library with yet another implementation.

A port is worse than a fork.

More chances for bugs.

More software to silently fall out of date.

More compatibility to lose.

** rotten apple slide *

LLVM in JavaScript

-- and all its other backends --

enables what I call the "Rotten Core" development model .

Programmers are very good at detecting rotten code.

It's quite easy, really.

Anything not written by they themselves is rotten.

More seriously, Rotten Core plus the LLVM stuff

means we don't have to make the platform

devils' bargain again.

We can abide with the embedded lessons

decades' old software libraries

provide our modern software

without abiding outdated APIs.

** Lynn tweet slide *

Lynn doesn't like GDAL's API,

but usable here means

**usable to her**

in her context.

What Lynn really wants is an abstraction layer,

-- an abstraction layer to the geodata abstraction library --

within the context of language and

programming idioms she already knows.

** rasterio slide *

rasterio is the most idiomatic Python

geospatial raster processing library.

It's all GDAL all the way down, but

it protects users from the most jagged bits

by hiding the necessarily complex GDAL API

and its use of C language idioms.

rasterio is Rotten Core and wraps the rotten-to-me stuff with

a convenient and idiomatic API

that has meaning within my own Python programming context.

With LLVM, Emscripten, and JavaScript,

we can start doing the same

for that environment too.

We did the same thing with the LiDAR compression

library. 5x less data transmission easily overcomes the 2x slower decompression performance. The tools are good enough to start doing this.

** GDAL.js slide *

Here’s Derek Dohler trying it on GDAL and PROJ. It’s super rough, but it appears gdalinfo works

I took over maintenance of the PROJ

library after Frank Warmerdam moved on.

It's everywhere, and it's going to continue to be

everywhere for an unpredictably long time.

The reason why is because it solves such a fundamental

problem of our domain

that once solved, doesn't

need to be solved over and over again.

None of its ports has feature parity.

PROJ is just now adding the ability

to do time dominant datum shifts.

Every feature, change, and bug fix radiates

to PROJ’s ports at time scales bested by the tectonic plates.

Software libraries that condense knowledge

of data formats and math

-- like GDAL and PROJ --

are going to be around for a very long time.

Longer than any of our careers.

Reflecting on Gerry Evenden for my PROJ talk

on Thursday made this fact very clear to me.

** diadem slide *

These libraries are heirlooms that will be patched up,


retargeted and

wondered about

with a kaleidoscope of abstraction layers

pasted on top of them to suit

new idioms.

It is folly to reimplement them

just to reach a new platform,

especially now that LLVM-enabled

Rotten Core programming can

work in JavaScript in the same way

it might in any native language.

** over and over slide *

JavaScript is just another platform,

and the tools are maturing enough that

we can treat it like any other one we've been building

boring geospatial software on for decades.

I hope you'll think of


Emscripten, and

Rotten Core

before you dive into

some brownfield reimplementation

of an open source library

that's been around for 20 years.

** cargo shorts slide *

Don't think all those pockets

aren't on there for a reason.