Python Intermediate Bootcamp

Sloppy code means a sloppy mind. Better code will make you faster and less error-prone. And you’ll spend less time rewriting it.
Bad code is hard to debug and comprehend. Better code will make it easier for you to add features or fix a deadline bug.
Bad code obscures what it’s trying to accomplish. Good luck explaining how you arrived at your result from a data set if your code is inexplicable and messy.
Programming, like journalism, is a craft. Getting better at your craft is an end unto itself.

What is bad code? Think about a time you’ve written code that made you itch. Why? Did it have bugs? Was it hard to read? Hard to change? Bad code, or code that makes you itch, typically has one or more code smells. [Talks about code smells]

What is good code? There are lots of ways to define “good” code, which could include judgments about code structure, how idiomatic it is, its efficiency, test coverage and many other variables. But let’s start with a simple definition that is easy to aspire to as a goal when we’re programming: Good code is READABLE code!

You don’t need to be a master programmer to decide whether your code is readable. You can tell just by looking at it.

Code Design Part I

Code smells and Functions

Overview of the long, gnarly script and discuss specific code smells in this script.

Shred this code

Exercise - Students form groups and chop up script (on paper) into smaller functions.

Code Review

Students present their solutions and we discuss differences in implementation. Compare their versions to ours.

Code Design Part 2

Intro to Modules and Packages

Overview of modules as namespaces for packaging code (data and functions/classes) and packages as collections of modules (with an __init__.py file). Whiteboard how we might organize our function-based script into more meaningful modules such as scraper.py, parser.py, summarizer.py

Poor man’s testing interlude

Before we start shuffling around code, how can we be sure we don’t break what we already have? Testing is the ideal way to do this for many reasons. But for today, we’ll use a simple file diff tool to compare the output of earlier versions of our code to later versions. This way we can have more confidence we’re not breaking things as we change the code.

Run python elex1/election_results.py
Run python elex2/election_results.py
diff elex1/summary_results.csv elex2/summary_results.csv
Manually modify elex2/summary_results.csv do another diff to see how file differences display

Module Mechanics

Module and package mechanics - Students individually follow along as we create below and experiment in python shell:

cd ~/pycar_intermediate/src/refactoring101
git fetch && git checkout pre-elex3
touch elex3/lib/scraper.py
Add a constant and hello_world function to elex3//lib/scraper.py

Demo import styles in python shell (open shell in top-level directory outside of elex3):

import scraper # blows up!!
Discuss packages and module search path very quickly (how we need __init__.py for packages and subpackages)
fully namespaced imports

import elex3.lib.scraper
print elex3.lib.scraper.SOME_CONSTANT
print elex3.lib.scraper.hello_world()

from...import… syntax

from elex3.lib.scraper import SOME_CONSTANT, hello_world

print SOME_CONSTANT
print hello_world()

from...import * syntax

from elex3.lib.scraper import *
print SOME_CONSTANT
print hello_world()

from elex3.lib.scraper import download; download()

(in python shell) Demo module.__dict__ and dir(module) to drive home point about modules as namespaces.

Module Search Path

A quick crash-course on the module search path in the course of wiring up the data summarization script to use our fancy new library code.

Code-Along

Update script stub with:

from elex3.lib.scraper import download
call dowload(path) inside main function

Run script and watch it blow up b/c elex3 is not on the search path. Explain mod search path and why the imports from python shell worked whereas this script failed

Exercise - Students (in pairs or individually) modularize the code, putting functions into scraper.py, parser.py and summary.py. Then add those functions to script. Goal is to have a working script that generates identical CSV output as elex2/election_results.py
Code review and discussion: A few students present their modules for discussion and we compare to our version.

Object Oriented Interlude

Why classes and OO?

What are classes and why should we care?

At most basic level, just another way to minimize complexity by logically grouping data and behavior.

Help tame code complexity and make it more readable!
Empower yourself to go beyond the docs. Even if you never use classes, you’ll have to read third party libs that use them. Understanding classes empower you to understand the actual implementation of the code and, when necessary, modify it to your needs. Examples of OO in the wild:

Django in general (models, class-based views)
Django class-based views
Pandas (e.g. DataFrame class)
Flask mini web framework

A few specific technical benefits:

Avoid duplicating code (this is deadly)
Help write reusable code (the good kind of lazy!)
Group code in logical ways (did someone mention readability?)
Make it easier to change code down the road
Simplify reuse by hiding complexity

Classes -- Bags of data and behavior

Intro - Classes are bundles of data and behavior with lots of functionality mixed in for taming complexity. Similar to modules they provide a way to namespace, or logically group code.
Exercise 1 - Write a simple Person class to demo class basics as a storage container for data

Note that Python 3 doesn’t require subclassing Object
Create Person instance and briefly discuss terminology: classes vs. instances vs. objects
Add and overwrite attributes on instance:

person.first_name = “Jane”
person.last_name = “Doe”

Add a method full_name” method with self as first arg. Illustrate “self” convention by calling:

Class method with instance as 1st param: Person.full_name(person)

Instance with implicit self: person.full_name()

Exercise 2 - Add Person.greeting method that combines salutation parameter with full_name method.

NOTE: return statement is required!

Customizing Classes

Python lets you pre-load objects with data upon creation, rather than having to manually assign attributes after the fact. This is done by defining a special method called __init__. Think of it as saying “create this class and initialize it with the following attributes before returning that instance to me.”

Explain __init__ as one of the double-underscore “magic” or special methods in Python and benefits of usage.
Rewrite Person with __init__ that requires first_name and last_name
Create Person instance and show how:

attributes that we previously assigned manually to instance are now available upon instantiation
an error occurs if we don’t supply these params on creation (a little built-in data integrity!).

Code Design Part 3

Reset code to a clean state and copy over libs from elex3 as a starting point.

cd ~/pycar_intermediate/src/refactoring101

git checkout pre-elex4
cp elex3/lib/* elex4/lib/

Modeling with Classes

Review current logic in parser.py and summary.py and draw up list of data attributes for new Candidate and Race classes (we can decide downstream if these should be data attibutes or methods)

election date
total votes for election
candidate name
vote count
etc.

Coding the Classes

Implement Candidate class, per the Refactoring101 tutorial.
Implement the Race class, per the Refactoring 101 tutorial.

Integrating the classes

Update the parser.py and summary.py to use the Candidate and Race classes
Copy over the script from elex3 to elex4

cp elex3/scripts/save_summary_to_csv.py elex4/scripts/
Update the script’s import statements in elex3 to use elex4 as top-level namespace
Run script and diff output to test class-based code changes

Code review

Compare and contrast with final implementation. What other changes could we make to improve the quality of this code? For example, making a Parser class with methods such as _get_or_create_race method? What else?

OO Interlude 2

Inheritance

Intro: Define broadly as a way for classes to inherit behavior from one or more ancestor classes. We’ll use example of a Person class that might serve as base for a Candidate and Contributor.
Exercises: Follow steps in Person class exercises.

Composition et al.

A quick mention of composition, multiple inheritance and other object-oriented concepts we won’t get into but which students should be aware of for future exploration.

Code Design - Final Thoughts

Intro: Overview of final implementation’s code organization and the language tools we used to accomplish our design.

Review and discuss progression from a single, unmanageable mess to a cleaner, more modular design.
Language features we didn’t discuss that can further assist with writing clean code (i.e. What’s Next?)

Q&A: Are there still warts on our design? What further changes would students make?

Testing and other good habits

Intermediate 0.1 to 0.2

0.1 is writing code that’s properly structured and readable. The next step is gradually adopting best practices to make it easier to manage code, deploy programs, and foster collaboration.

Keep in mind this is a quick overview of a lot of tools and best practices. Don’t try to learn and apply them all at once. That’s a recipe for frustration. Adopt them organically over time, as you face new challenges. You’ll know when you need them.

Version Control - Stop whatever you’re doing right now and use git! Yes, git.
Debugger and stack traces (learn to love them)
Tests - demo a few unit tests
Environmental hygiene

virtualenv - sandbox your environment
pip - specify your dependencies and make install a cinch

Deployment and provisioning (often a blurry line between the two). A smattering of tools to consider:

Distribution - package code for distribution with setup.py or, nowadays, wheels

How To Get Better

Write code.
Read lots of other code, especially major libs (requests, Tastypie, Django).
Keep this by your nightstand. Especially the Structuring Your Project section.
Read books to add depth of knowledge on techniques

Not just about syntax (Learning Python, Pro Python etc.) but also about solving problems in specific contexts (Programming Python, e.g.)
How to read: skim to become aware of techniques, practice with exercises, apply techniques to master them

Read books (not necessarily about Python) to add knowledge about software as a craft

Design Patterns
Refactoring

Make open source contributions