1 of 42

What Django Deployment

Is Really About

James Walters

2 of 42

Now, I want to preface this talk by bringing up Flask for just a moment, or Pyramid or just about any other Python web framework. I’m a Django developer, and this is a talk on Django specifically, but I want to emphasize that a lot of the broader principles that we’re talking about here will apply to other frameworks as well. The things that we’re talking about, when you get down to it, really apply to running a web app generally, including Python apps through WSGI. So while I can’t comment on the precise way something might be done in Flask, if you’re a Flask developer, you’ll probably understand how you might apply this to your situation. But for the rest of this talk, we’re going to address Django specifically.

So, let’s talk about what Django deployment is really about.

How many of you think that WSGI is something you order at the bar?
How many of you think a gunicorn is some sort of ballistic horse?
How many of you have never deployed a Django project before? Maybe you’re a beginner and you’ve never done it, maybe someone else is the deployment person at your organization, etc.

And how about the question of the hour…(click)

3 of 42

Is Django Hard to Deploy?

🤔️

James Walters - james.walters.click

4 of 42

The Penguin in the Room

James Walters - james.walters.click

5 of 42

Four Main Concerns

Static Files
Database
WSGI Server
Web Server

(and a couple of other things)

James Walters - james.walters.click

What we are going to do today is boil deployment down to a handful of main concerns.

Because the problem with deployment, I think, is that we teach people how to build apps, we focus on getting to know Django and using it to build out a project, and then when we get to the end of that what often happens is we give someone a deployment recipe, or a list of steps to follow. But the steps that worked yesterday might not work today, or the provider you used might not work for me. Or even if the recipe does work, maybe I got my app online, but I didn’t really learn anything about deployment. I don’t understand anymore than I did before about what’s going on in the process. And there’s a lot going on in the process, there are a lot of moving pieces to deployment.

So I want to break down deployment into four main concerns that you need to think about when you’re taking your app from your code editor to the web.

The first (click) is static files.

6 of 42

Static Files 📁️

James Walters - james.walters.click

7 of 42

But {% static %}, right?

James Walters - james.walters.click

8 of 42

index.html

about.html

news.html

posts.html

www.yourwebsite.com

Apache / nginx

James Walters - james.walters.click

9 of 42

template.html

www.yourwebsite.com

Apache / nginx

Database

Django

James Walters - james.walters.click

10 of 42

template.html

www.yourwebsite.com

Apache / nginx

Database

Django

style.css

STATIC_ROOT

script.js

. . .

James Walters - james.walters.click

11 of 42

Static Files

Once you deploy, the production web server needs to handle this.
Once you’ve set STATIC_ROOT, you have to run manage.py collectstatic to actually collect your static files inside the STATIC_ROOT directory.

You have to do this whenever you add or change any of your static files.

James Walters - james.walters.click

Django actually has been handling static files for you in development, because manage.py runserver is the only web server available.

12 of 42

But Whitenoise, right?

It’s great for situations where you can’t directly configure your web server
Also, excels when serving your site from CDNs–automatically handles these HTTP headers for you!

The cache-busting that Whitenoise can do for you is a real plus if you rely some JS running client-side to handle any core functionality

Whitenoise is a middleware that intercepts the request and serves up the static file, skipping the rest of Django’s (comparatively) slow request cycle

James Walters - james.walters.click

What about Whitenoise?

If you’re not familiar with Whitenoise, it’s a middleware that you can add to Django that intercepts the request in the middle of the cycle and if the request is for a static file, Whitenoise will interrupt the rest of that slow request cycle, and serve up the static file immediately. (click)

It’s a great option if you’re in a situation where you can’t directly configure your web server, like maybe your virtual machine is handled by your infrastructure team, and you can only push code to it. (click)

Where Whitenoise really excels is if you’re serving your site from a CDN (or content delivery network). In that case, it’s beneficial to add HTTP headers for things like caching and compression, and Whitenoise takes care of all that for you without having to think about it. (click)

Also, that cache-busting that Whitenoise can do for you is a lifesaver if you rely on any JS running on the client side to do any important business logic for you. I work for a call center, and a colleague and I rebuilt our agents’ knowledge base web app for handling calls, and the app had to wire into our phone system through a JavaScript API. There came a point where I needed to make some important changes to that code, but I couldn’t afford to have people using outdated JS files from their browser cache, that would have rendered them unable to control their call. I had to make sure that when that code deployed, it was requested fresh by everyone’s browsers, and in order to do that I had to set up Whitenoise with cache headers.

So Whitenoise really is a great way to solve all these problems. There’s a reason that it’s virtually the default choice at this point for the average Django developer.

But if you’re not on a platform-as-a-service, it’s also worth taking a look at writing the web server configuration yourself, for at least a few reasons. (click)

13 of 42

But Whitenoise, right?

If you have access to your web server, consider writing an nginx/Apache config–it’s not that hard!
You made a web app–shouldn’t you understand how a web server works? 🤔️

James Walters - james.walters.click

https://realpython.com/django-nginx-gunicorn/#serving-static-files-directly-with-nginx

14 of 42

What about user uploaded files?

Whitenoise won’t work here
Eventually you’d probably want to set up a third party storage backend, like an Amazon S3 bucket, for user uploads.

Similar process, but use MEDIA_ROOT / MEDIA_URL instead of STATIC_ROOT / STATIC_URL in settings.py (see Django documentation)

James Walters - james.walters.click

Django calls these “media files”, stuff that people other than you post on your site. Profile pictures, file attachments, etc.

The way you set this up is pretty similar to static files, instead of STATIC_ROOT and STATIC_URL you’ll use MEDIA_ROOT and MEDIA_URL in your settings.py, and again (click) Whitenoise won’t help you here, Whitenoise only deals with static files. So you’ll need to know how to set up your deployment situation to find your store of user-uploaded media files. (click)

If you’re in production, what this most likely will look like is configuring a cloud object storage backend, like Amazon S3, for those files to go live in. The rule of thumb is you don’t want to store those on the same machine that your app is running on, you don’t want to permit people to upload arbitrary data onto your production machine, so a little separation here is hygienic.

(click)

15 of 42

Database 💾️

James Walters - james.walters.click

16 of 42

But SQLite, right?

In deployment, SQLite may not be suitable (though it's more capable than you're often led to believe)
Generally if you're using some sort of cloud provider, you'll be connecting to a remote database that's managed for you.

James Walters - james.walters.click

https://www.youtube.com/watch?v=yTicYJDT1zE

But SQLite, right? It’s been doing great for me here in development, why can’t I use it in deployment?

And that’s a question that we’ll leave for the database admins among us. Opinion is in fact shifting on this topic, it’s possible to configure SQLite in ways that will scale quite far, I’ve linked to Tom Dyson’s talk from DjangoCon Europe 2023 about Using SQLite in Production where he makes a compelling case for this, Simon Willison is also using SQLite behind his Datasette project. But the point here is not so much what database you use, but that your database may well be different in deployment than it is in development, and that’s something you have to think about in this process.

Generally, if you’re using some sort of cloud provider, you’ll be connecting to a remote database that’s managed for you. That means you’ll have a URL for the database, a port, username, password, this kind of stuff, and all of that will live in your settings.py. (click)

17 of 42

But SQLite, right?

This probably won’t be different for your app in any material way–the important difference will be that you'll need to configure connection details in your settings.py.

SQLite keeps everything in one single file. Since you don't "connect" to it, you don't have to handle connection details. Your DATABASE section in settings.py is pretty simple.

James Walters - james.walters.click

18 of 42

settings.py

…

DATABASES = {

"default": {

"ENGINE": "django.db.backends.sqlite3",

"NAME": "mydatabase",

}

…

DATABASES = {

'default': {

'ENGINE': 'django.db.backends.postgresql'

'NAME': 'mydatabase',

'USER': 'mydatabaseuser',

'PASSWORD': 'mypassword',

'HOST': '127.0.0.1',

'PORT': '5432',

}

…

James Walters - james.walters.click

19 of 42

Database

The point here is not to compare databases and tell you which one to choose–the point is to show that your database situation will almost certainly be different in deployment, and you need to be aware of that.

Once you’ve got your connection details set up, you’ll want to run manage.py migrate to set up your database for the first time.

James Walters - james.walters.click

20 of 42

WSGI Server 🥃️

James Walters - james.walters.click

21 of 42

But manage.py runserver, right?

James Walters - james.walters.click

22 of 42

www.yourwebsite.com

Apache / nginx

Django

James Walters - james.walters.click

23 of 42

www.yourwebsite.com

Apache / nginx

Django

WSGI

Server

Gunicorn

Waitress

uWSGI

wsgi.py

James Walters - james.walters.click

The answer is W-S-G-I (or Web Server Gateway Interface). It's usually pronounced whizz-ghee or whiskey. The way this works is when the web server gets a request, it passes it over to a WSGI server (like gunicorn or Waitress or uWSGI, you might have heard these names before, this is what they are, they’re WSGI servers). That WSGI server runs a WSGI callable for your application. How on earth do I make a WSGI callable, you ask? (click) Django already did it for you on the first day when you ran startproject—it's in the wsgi.py file.

So you set up your WSGI server and tell it, whenever you get a request, run it through this Django app’s WSGI callable. Then, the WSGI server will run requests through your app, and after generating the appropriate page, pass that response back to the web server to hand off to your user.

To configure this, you’ll need to consult the documentation of your chosen WSGI server. But again, the point here is not to tell you which one to pick, but that you have to pick one. (click)

24 of 42

Web Server 🕸️

James Walters - james.walters.click

25 of 42

Web Server

Apache and nginx are your two main options, both are great.
RealPython has a good article on how to configure a Django project to use gunicorn through nginx.

James Walters - james.walters.click

https://realpython.com/django-nginx-gunicorn/

26 of 42

But do you even need to configure the web server yourself? 🤔️

It depends on your deployment choice…

James Walters - james.walters.click

27 of 42

VPS

Examples include: Linode, Digital Ocean
It'll be a very manual setup process, with little hand holding. But, you'll understand every step of the way.
If you go this route, you'll be responsible for everything about the server: restarting it if it goes down, tracking and rotating log files, managing security settings and updates, etc.

A virtual private server (VPS) is a virtual machine running in the cloud.

James Walters - james.walters.click

You might be running your app on a bare metal server in your home lab or at your company’s data center, but for most of us most of the time, we’re going to be deploying in the cloud, and the question will be VPS vs PaaS. So let’s get into what those mean.

VPS stands for virtual private server, and this is a virtual machine running in the cloud. It’s basically a rented computer. (click) You can get these from lots of providers, a couple of examples are Linode or Digital Ocean. (click) A virtual machine is just that, a virtual machine, so this is like setting up your own server, except the hardware is rented from a cloud provider. So, it’s a very manual setup process, you have to configure the entire operating system. But the benefit of this is that if you go down this route, you’ll understand all the layers that support your app. (click)

The disadvantage is that you’re responsible for everything about the server. If it goes down, you have to reboot it. You have to rotate out log files before they fill up your disk. You have to understand and set security settings, stay on top of updates, etc. etc.

The alternative (click) is platform as a service, or PaaS.

28 of 42

PaaS

Examples include: Heroku, PythonAnywhere, DigitalOcean, Fly.io
The provider will abstract away the underlying computer, and you just focus on your app.
You get your code uploaded, provide some details about hooking up a database, static files, where the script to execute is, etc.
You don't have to worry about the server-y stuff like worrying about updates, checking logs, understanding security settings, or configuring the web server.

A platform-as-a-service (PaaS) is kind of different.

James Walters - james.walters.click

29 of 42

(Soft) Verdict?

James Walters - james.walters.click

^{✅️ PaaS}

30 of 42

James Walters - james.walters.click

31 of 42

Django's Deployment Checklist ✅️

James Walters - james.walters.click

32 of 42

Django’s Deployment Checklist

For example, it will warn you if DEBUG = True.
To use it, just run manage.py check --deploy

Django has a handy built-in automated tool that will look over your settings file and point out any settings with values that aren’t appropriate for deployment.

James Walters - james.walters.click

33 of 42

manage.py check --deploy

$ python manage.py check --deploy

System check identified some issues:

WARNINGS:

?: (security.W004) You have not set a value for the SECURE_HSTS_SECONDS setting. If your entire site is served only over SSL, you may want to consider setting a value and enabling HTTP Strict Transport Security. Be sure to read the documentation first; enabling HSTS carelessly can cause serious, irreversible problems.

?: (security.W008) Your SECURE_SSL_REDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS.

?: (security.W009) Your SECRET_KEY has less than 50 characters, less than 5 unique characters, or it's prefixed with 'django-insecure-' indicating that it was generated automatically by Django. Please generate a long and random value, otherwise many of Django's security-critical features will be vulnerable to attack.

?: (security.W012) SESSION_COOKIE_SECURE is not set to True. Using a secure-only session cookie makes it more difficult for network traffic sniffers to hijack user sessions.

?: (security.W016) You have 'django.middleware.csrf.CsrfViewMiddleware' in your MIDDLEWARE, but you have not set CSRF_COOKIE_SECURE to True. Using a secure-only CSRF cookie makes it more difficult for network traffic sniffers to steal the CSRF token.

?: (security.W018) You should not have DEBUG set to True in deployment.

?: (security.W020) ALLOWED_HOSTS must not be empty in deployment.

James Walters - james.walters.click

34 of 42

django-simple-deploy 😃️

James Walters - james.walters.click

35 of 42

django-simple-deploy

From Eric Matthes, author of Python Crash Course
Aims to be a utility that you can run against your code, and it will automatically handle all the steps needed to get your code deployed.

James Walters - james.walters.click

https://django-simple-deploy.readthedocs.io/en/latest/

36 of 42

django-simple-deploy

Helps a newcomer get their code online, then they can compare the changes it made to their code settings and learn how exactly it went about it.
Currently supports deploying to Fly.io, Platform.sh, and Heroku. More platforms should be on the way.

James Walters - james.walters.click

https://django-simple-deploy.readthedocs.io/en/latest/

Obviously, this is great for newcomers, not only because they just finished their project and this helps them get it onto the web expediently, but also because if you have git, then you can diff all the changes it made to your code in order to get an idea of the sorts of things you need to change or configure in order to get your app deployed.

It’s not just targeted at newcomers though, Eric envisions this being useful to people who want to experiment with different cloud providers, as well as to those cloud providers themselves, it certainly helps simplify getting your customers online if you can say, “by the way, you can run django-simple-deploy and get online with our platform.” So very cool project, you should check it out.

Currently, Fly, Platform.sh and Heroku are supported, and more are planned, so watch this space. (click)

37 of 42

django-production 🏭️

James Walters - james.walters.click

38 of 42

django-production

From Peter Baumgartner at Lincoln Loop
Aims to provide sane defaults for getting your Django project into a production deployment setting

James Walters - james.walters.click

https://github.com/lincolnloop/django-production

39 of 42

django-production

whitenoise - for serving static files
django-environ - for reading settings from environment variables
django-webserver[gunicorn] - for running the webserver via manage.py
django-alive - for a health check endpoint at /-/alive/

James Walters - james.walters.click

https://github.com/lincolnloop/django-production

From README:

Some highlights of what it actually does for you are:

It installs Whitenoise to handle static files,
It installs django-environ for reading environment variables, something we haven’t touched on today but is often a factor in getting deployed
Installs a package to allow you to run your webserver from manage.py. That defaults to gunicorn, that might not be a great choice if you’re developing on Windows, so maybe look into using WSL if you go this route
It will add a health checkpoint, which is important for integrating into system monitoring, if you’re using that,
And it will add stuff to your settings.py file for all of these things, right out of the gate.

So, this ticks a couple of the boxes we’ve talked today, it handles static files and the WSGI server pieces for you, and that’s sort of half the battle right there. So this project is worth looking at as well.

40 of 42

The Whale in the Room

41 of 42

Docker Pros

Docker Cons

Isolate application environments

No more “well it works on my machine”

This means you can deploy multiple apps to the same machine
Streamline deployment

docker run
docker-compose up

Adds some complexity�
You still have to know everything we’ve talked about

(click) The first big advantage you get out of running a Docker container is that you isolate the application environments. I know that Will Vincent is a big fan of this for development. (click) This means you’re always using the same Python version, the same Django version, there’s no more spooky action at a distance, no more “well it works on my machine”, because you’re passing around not only the code but the environment it runs in, everything here is a lot more consistent and reproducible.

(click) This also means that it’s really easy to deploy multiple apps on the same machine, if that’s something you need to do. You can deploy multiple apps on one machine without docker, but that starts to get really messy. It’s much nicer when all the apps just live in their own little box and can’t really touch each other, that keeps your underlying server nice and clean. And it’s worth mentioning that some of the platform as a service options will do exactly this, they’ll take your code and bundle it up inside a docker container.

(click) (click) (click) The second advantage that you get is the deployment process gets condensed down to a docker run command, or a docker-compose up command. You get to wire up the application and the database and the web server in configuration files or scripts, and then you just run them. So that’s got a strong appeal.

But there are some drawbacks to this, and really, it just boils down to (click) added complexity. Docker is now another thing that you have to understand and wrap your head around. And although it can be fairly straightforward once you’ve learned it, it adds a lot of complexity for the beginner that you really may not need to bring on your plate right now. It can be confusing to understand how you can run commands from inside your container, or how you wire up containers to talk to each other, and containers are ephemeral by default so if you want to save your data somewhere you’ve got to learn about docker volumes.

Michael Kennedy recently said on a podcast commenting on deployment that if he were dealing with a total beginner he would show them how to deploy from scratch on Linux, and I agree. Docker is just Linux, but inside a little box. So if you know how to do it on Linux you'll be able to do it in a docker container. But if not, then you won't, which brings me to my next point (click) which is, you still have to know everything we talked about. Docker doesn’t give you a shortcut here. You still have to deal with static files, your database, your WSGI server, and probably your web server as well. You’ve just added the abstraction that these things no longer run on your computer, they run inside tiny slices of your computer. And the advantage to that is the slices are mostly consistent and mostly keep to themselves and you can start the slices with a single command. That’s nice, but it doesn’t abstract away the essentials we’ve talked about today.

42 of 42

Wrapping Up 🎁️

See my blog james.walters.click for links to these slides, and the resources mentioned.

Github: github.com/iamjameswalters

Mastodon: @jameswalters@fosstodon.org

Email: jameswalters@hey.com

Thank You!