Programming with AI

How LLMs are changing the way we write code

​

http://bit.ly/uc-dss-2023-ai-code

Ethan Swan

UC Data Science Symposium, 2023

AI tools speed up coding by between 1.5x and 10x

That’s what we’re here to talk about!

​

Biggest gains:

• Writing in unfamiliar languages
• Repetitive code

Little/no gain:

• Complex or novel business logic

About me

• Backend web development, data engineering, internal tool development
• Previously in data science
• Lots of Python, but aspiring polyglot
• Go, Ruby, Rust, Javascript, Scala, R

​

• Based in Chicago
• Work at ReviewTrackers (~100 person SaaS company)

Agenda

1. AI coding assistants overview
2. How does AI make coding faster/better?
3. Adapting your workflow
4. AI weaknesses

Overview of AI coding assistants

Types of AI assistants

We’re going to talk mainly about chat and autocomplete today, with a focus on the most popular offerings of both (OpenAI Chat GPT & GitHub Copilot)

AI Chat

• Conversational format
• Chat GPT has two tiers
• GPT-3.5 is free
• GPT-4 is paid, and much better at problems that require “understanding”

AI Autocomplete

• GitHub Copilot dominates market
• As you type, the assistant tries to guess the rest of the line
• Or even more than that!
• Trained on public source code on the internet

How does AI help with coding?

Learning new languages and frameworks

• Code for common, basic tasks can be filled in automatically (ask chat or write descriptive function signature)
• Ask chat assistants to translate from one language to another
• Writing a “plan” in a comment can be enough to help autocomplete figure out several lines of code

Boilerplate and repeated code is a breeze

• Common snippets like `with open(filename, ‘r’) as f: contents = f.readlines()` can often be guessed by autocomplete within a few characters
• Lower-level and more verbose languages are less cumbersome
• Tests in the same file often contain lots of similar setup and assertion code – LLMs can help
• Though you may need to make small tweaks after

Figuring out “best practices”

• Just ask a chat assistant!
• “What’s the modern best practice approach for building a small Python Rest API around a predictive model?”
• Wondering why one way is preferred? Ask that too

Adapting your workflow to AI

Structure is your friend

• This is the key insight
• LLMs are ultimately predicting the most likely text that follows an input, based on context.
• Structure is a type of context, and it helps go from *plausible* code to *working* code
• Because it knows how to chain together functions and objects based on their methods & attributes & signatures.
• Be consistent in your own code
• Consistent naming, consistent patterns
• Use naming and patterns that are common in the wider world
• Naming matters more than ever – LLMs can guess how you’ll use a variable based on its name

Static typing

• Static typing helps LLMs a lot (again, structure)
• Statically-typed languages restrict what can be passed into functions, what attributes can be accessed, etc.
• So in cases where typing is option (Python), use it!
• Use mypy to check they’re actually accurate

Don’t worry so much about DRY

• Boilerplate and repetition can be written by LLMs instantly
• DRY still matters because updates to the code will need to update multiple places
• But some languages are notorious for verbosity and it’s not such a big deal now
• explicit typing in C and Java, `if err != nil` in Go

Specific strategies: writing functions and classes

• Give the function a clear name.
• Write out its signature (params & return value, typed).
• Then let the model start guessing…
• If it's going in the wrong direction, add a comment at the top explaining roughly what you want

Specific strategies: writing tests

• Give the test a good name.
• The model might fill it in from there, but more likely you'll need to write the code that "sets up" the thing you want to test/assert.

Specific strategies: writing tricky code

• Leave a comment that isn't a request (find this, or do this) but instead a note about the code that will exist.
• Like… # The below code finds the N largest records by the "count" field.
• NOT: # Write the code to get the N largest records by the “count” field
• The LLM can often figure out *how* to do that task once it's been described.
• Remember that LLMs are remembering what code usually follows certain comments.

Specific strategies: writing code for a common task

• Just ask chat to write it for you – define the task clearly
• Provide the function signature and return value if its first try doesn't fit your needs quite right.
• E.g. Write the python code to sort a pandas DataFrame by its “count” column, ascending, with nulls first.

Specific strategies: starting a new project

• Again, just ask chat!
• E.g. I'm starting a scala project for the first time. What's the simplest way to initialize a new project? What should my file structure look like?
• Ask follow ups as needed.

Specific strategies: “tuning”

• For the previous strategies, the generated code will only be perfect about 30% of the time
• Depending on how complex the request was
• You need to read or at least skim over the code yourself
• Don’t look for typos – it’s good at those. Look for misunderstanding of the problem
• Wrong framework, outdated syntax, etc.
• Either modify the code yourself if you know how, or paste in chat and explain what’s wrong

Beyond coding: tech choices

• Always lean toward common tools and frameworks for a given task
• Even more than usual!
• LLMs have seen variations of solving problems with Pandas many times; not so for Polars
• Be more open to refactors and language/framework switch-outs
• The LLM can help you translate from one language/framework to another
• Coders can “write” new languages more efficiently than before
• Autocomplete can generate a large number of tests to get coverage before a refactor

Beyond coding: career

• Jobs that don’t let you use AI coding assistants are going to be a career drag
• Using AI is going to become the norm rapidly – productivity gains are too big
• Being skilled with it will be important in interviews
• Also: once you get used to using it, coding without it feels like typing with mittens on

Beyond coding: hiring & management

• Interview more for ability to solve business problems and think through complex problems
• Less based on familiarity with certain languages/frameworks
• It’s vital to get these tools approved for your team
• Productivity gains
• More important: career prospects for recruiting and retention
• Corporate security teams will be inclined to say no to this kind of thing; you need to push hard for its approval

Artificial stupidity: AI weaknesses

Novel business logic

• AI can’t read your mind and won’t guess at specific business logic unless it’s been coded before
• But you can explain it in detail to a chat assistant
• If you don’t guide it carefully, it’ll just keep guessing wrong
• Might even slow you down

Bias toward older tech

• New frameworks and language versions are relatively rare in the training set
• For example, GPT-4 doesn’t know anything since 2022
• Frameworks with recent overhauls (sqlalchemy, pydantic) will be unknown to AI assistants

Garbage in, garbage out

• If code structure and naming don’t “remind” LLMs of something they’ve seen before, they can’t help you

Skill decay

• Sad as it is – you’re going to get worse at coding (as we think of it)
• Unlikely to have the same recall of specific functions and patterns
• But you’ll get more done, and it’s worth the tradeoff
• Though if you really like coding, you might disable it sometimes
• It’s a lot like switching from a minimal text editor to a powerful IDE

Internet reliance

• I recently tried to code on a plane and found myself surprised at my own limitations
• Paid coding LLMs are probably going to be server-side for a while

Last thoughts

AI is here to stay

• AI chat and autocomplete are already revolutionizing coding
• The $10-$30/month subscriptions are tiny compared to productivity gains
• Assistants thrive on structure, clear descriptions, and problems that are similar to what they’ve seen before
• Invest in them fast
• ICs: get experience with them
• Managers: get them approved and hire differently