I Don't Like Notebooks

Joel Grus (@joelgrus) #JupyterCon 2018

how we

ended up

here

@joelgrus #jupytercon

the idea was received pretty warmly

@joelgrus #jupytercon

about me

@joelgrus #jupytercon

research engineer @ Ai2

@joelgrus #jupytercon

know a few things about python

@joelgrus #jupytercon

know a few things about data science

@joelgrus #jupytercon

know a few things about software engineering

@joelgrus #jupytercon

livecoder

@joelgrus #jupytercon

blogger

@joelgrus #jupytercon

...

podcast co-host

@joelgrus #jupytercon

what I assume about

you

@joelgrus #jupytercon

a couple of caveats

@joelgrus #jupytercon

@joelgrus #jupytercon

things I do like about notebooks

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

things I don't like about notebooks

@joelgrus #jupytercon

notebooks have tons and tons of hidden state that's easy to screw up and difficult to reason about

@joelgrus #jupytercon

simple example

@joelgrus #jupytercon

if you look at the numbers, it's clear those cells weren't even executed in order!

@joelgrus #jupytercon

if you look at the numbers, it's clear that something else was run between the first two lines

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

one more

@joelgrus #jupytercon

...

oops!

@joelgrus #jupytercon

notebooks are difficult for beginners

@joelgrus #jupytercon

Start off by creating a notebook.

@joelgrus #jupytercon

(hidden state complications are not obvious)

@joelgrus #jupytercon

this is most people's experience of how code works

the ability to run code snippets in arbitrary order is weird and unintuitive

@joelgrus #jupytercon

beginner tutorials are cavalier about (or silent on) hidden state

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

as notebook experts, the problem here is obvious to you

​

(also this is a really simple example)

​

for beginners, with dozens of cells and more complex code, this is utterly confusing

​

(I know this because they come to me with their problems)

@joelgrus #jupytercon

in fact, my original angry tweet was prompted by the [large number]^th occasion of me helping a newcomer to python who used a notebook because they heard that was *the thing to do* and who found "python" bewildering 100% on account of messed up hidden state in their notebook due to normal beginner sloppiness and out-of-order execution

@joelgrus #jupytercon

Lots of beginners use notebooks; clearly it can't be that difficult.

It's not that it's insurmountably difficult, it's that the out-of-order execution makes learning Python more confusing than it needs to be.

@joelgrus #jupytercon

notebooks encourage bad habits

@joelgrus #jupytercon

@joelgrus #jupytercon

apparently this was a slight misrepresentation of the "tip", although I didn't learn that until after I started a Twitter war over it

@joelgrus #jupytercon

Rigorous software engineering isn't that important, I'm just experimenting!

You mean you're just

doing science?

@joelgrus #jupytercon

I just want to see if my model works before I put it into production.

Don't you need to write correct code to make sure it works?

@joelgrus #jupytercon

anti-pattern: the need for speed

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

"Here's some code I wrote, but I don't want my client or decision maker to see"

@joelgrus #jupytercon

notebooks discourage good habits

@joelgrus #jupytercon

modularity

@joelgrus #jupytercon

testability

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

notebooks are way less helpful than my text editor

@joelgrus #jupytercon

some things are easier demonstrated

@joelgrus #jupytercon

Just run the cell first. And then just press TAB.

@joelgrus #jupytercon

Just replace the parentheses with a question mark. And then just execute the cell

This also "uses up" a computation, so either I have to leave my ignorance there or have "out of order" cells

@joelgrus #jupytercon

You may not want to use a filename as your variable name, but some people might!

@joelgrus #jupytercon

...

@joelgrus #jupytercon

...

I found an extension called "hinterland" that does some of these, but...

• questionable autocomplete behavior���

​

​

​

​

• still can only autocomplete things that have been executed

• which means still can't handle type hints or with blocks

enter

@joelgrus #jupytercon

...

@joelgrus #jupytercon

those are the tools that help me be productive and write good code

@joelgrus #jupytercon

You just need to use the next-generation JupyterLab.

@joelgrus #jupytercon

notebooks encourage bad processes

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

notebooks HINDER

reproducible + extensible science

@joelgrus #jupytercon

@joelgrus #jupytercon

notebooks are a recipe for poorly factored code

@joelgrus #jupytercon

idea:

train a neural network to generate music

​

this is the true story of an idea that my co-workers and I were kicking around at lunch a couple of months ago, there was more to the idea but this was at the core

@joelgrus #jupytercon

"standing on the shoulders of giants, etc., etc."

@joelgrus #jupytercon

😬

was hoping for model.py or models.py or something

​

was hoping for model.py or models.py or something

@joelgrus #jupytercon

nope

pytorch 0.3? 0.4? 0.4.1?

​

I suppose I could guess based on the commit date

@joelgrus #jupytercon

an aside

@joelgrus #jupytercon

model definition

model instantiation (with hard-coded parameters)

more parameters

hard-coded paths

more parameters

training loop

@joelgrus #jupytercon

• This code is pretty much impossible for me to use
• First, I have to look at the notebooks, manually install all the dependencies, and hope I got the right version
• Then either:
• I can run it the exact same way on the exact same data (at the exact same paths!), or
• I can copy the notebook and try to figure out how to modify it (forcing me to work in a notebook), or
• I can do a lot of work to copy and paste the right parts into a module that I can import into my own code

You know, there is a tool called nbconvert

@joelgrus #jupytercon

(yes, I am aware of nbconvert)

$ jupyter nbconvert music_generation_training_nottingham.ipynb --to python

• still mixes "library" code and "execution" code

​

• still have to go through and perform surgery

​

• still no requirements.txt / unit tests

​

• still few comments :(

@joelgrus #jupytercon

to be perfectly fair

• this is someone's fun project
• kudos to them for sharing what they did
• likely they thought they were being helpful by providing notebooks
• what they did was make it semi-possible (but not easy) for me to repeat their exact same code (but only if I have my data at the exact same paths and the right versions of all their dependencies, ¯\_(ツ)_/¯)
• while making it very hard for me to build new things on top of their code
• again, lots of non-notebook code has these same issues, it's just that notebooks implicitly encourage a workflow that has these issues

@joelgrus #jupytercon

but imagine...

@joelgrus #jupytercon

but imagine...

• a requirements.txt (possibly even a Docker image) indicating exactly what dependencies I need
• clean, parameterizable load_data.py and model.py with good documentation and unit tests (but I repeat myself)
• an easy way to programmatically specify model parameters, data paths, and so on
• a run_experiment.py script that replicates exactly the original result, and that can be tweaked in obvious ways to run novel variants

@joelgrus #jupytercon

@joelgrus #jupytercon

notebooks make it hard to collaborate across mediA

@joelgrus #jupytercon

the scene: pytorch slack, #beginner channel

@joelgrus #jupytercon

"It never hurts to help!"

@joelgrus #jupytercon

I imagine it took you a few tries to produce that, a notebook would have made it easier

@joelgrus #jupytercon

ok, I guess I could do that IN a notebook

@joelgrus #jupytercon

but how to copy and paste into slack?

@joelgrus #jupytercon

select all -> copy -> paste

@joelgrus #jupytercon

select cells -> edit -> copy cells -> paste

doesn't seem to do anything outside of the notebook

​

that is, when you go to Slack, there's nothing in the clipboard

@joelgrus #jupytercon

select cells -> copy -> paste

@joelgrus #jupytercon

basically, I have no idea how to copy and paste code + outputs from a notebook into slack

@joelgrus #jupytercon

as far as I can tell, in JupyterLab even the "ugly" versions don't work!

this may seem frivolous, but I spend a lot of time debugging people's python issues via slack

@joelgrus #jupytercon

same thing for github issues / code reviews

@joelgrus #jupytercon

one other nuisance that I discovered when working on this example!

@joelgrus #jupytercon

kernel -> restart and run all

error was expected

​

(and is part of what I want to demonstrate)

​

but halts execution

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

notebooks make it easy to teach poorly

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

notebooks make it hard for me to teach well

@joelgrus #jupytercon

this is from a real book

written by me

Data Science from Scratch

available wherever books are sold

@joelgrus #jupytercon

2^nd edition being written as we speak

OK, not exactly as we speak.

You should make a notebook version of your book.

Sure, I'll give it a try!

@joelgrus #jupytercon

this is from a real book

written by me

Data Science from Scratch

available wherever books are sold

@joelgrus #jupytercon

2^nd edition being written as we speak

OK, not exactly as we speak.

Well, your book shouldn't split the class into multiple pieces like that

@joelgrus #jupytercon

...

You'd rather I dump several pages of code at the reader all at once, then refer back to it bit by bit?

@joelgrus #jupytercon

It's not just me!

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

this was also suggested

@joelgrus #jupytercon

every one of these requires presenting unnatural code to my readers, solely to appease the notebook gods

@joelgrus #jupytercon

simpler examples too

this is not a real variable, which means this snippet isn't executable

​

this example would not be improved by actually defining that variable

what I do instead

@joelgrus #jupytercon

markdown tutorials

and

docs

@joelgrus #jupytercon

vscode + ipython console

@joelgrus #jupytercon

demo

(time permitting)

@joelgrus #jupytercon

how you could win me over

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

IDE-style autocomplete

@joelgrus #jupytercon

real-time typechecking and linting

@joelgrus #jupytercon

a better story around dependency management

@joelgrus #jupytercon

first-class support for refactoring code out of notebooks into modules

@joelgrus #jupytercon

the reality is

• you're not going to provide me all these things
• I'm not going to switch to notebooks
• but hopefully I've challenged you to think about
• how your tools limit you
• how to write better software
• how to teach better
• how to practice better science
• how different workflows could make your life easier
• some ways that you could make notebooks better

@joelgrus #jupytercon

in conclusion

@joelgrus #jupytercon

please come to my next talk:

"I don't like that slideshow program where sometimes the next slide is RIGHT and sometimes the next slide is DOWN"

@joelgrus #jupytercon

thanks for coming!

• will tweet out slides: @joelgrus
• check out my book Data Science from Scratch
• very infrequently updated blog: joelgrus.com
• podcast: adversariallearning.com
• livecoding videos: youtube.com/user/joelgrus
• tell me why I'm wrong: joelgrus @ gmail
• and of course I have a SoundCloud 😂😂😂: soundcloud.com/joel-grus

@joelgrus #jupytercon

appendix (rejected memes)

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

@joelgrus #jupytercon

UNTITLED25.IPYNB

UNTITLED24.IPYNB