1 of 112

Greening Digital Workshop:

�Set up your django app with green coding metrics

GREENWEB.ORG

Chris

Adams

2024.06.07

DjangoconEU 2024

1

GREEN WEB FOUNDATION

2 of 112

Hello, I’m Chris!

My background:

Loco2 - Low CO2 Travel in Europe by train

A.M.E.E (Avoid Mass Extinction Engine) - CO2 calculation as an API

Icebreaker One - data infrastructure for a net zero future

Spend Network - direct public spending for net zero

Green Software Foundation - Policy chair

Branch Magazine - editor

Green Web Foundation - make the web green

Environment Variables podcast - co-host

2

GREEN WEB FOUNDATION

3 of 112

Housekeeping

GREEN WEB FOUNDATION

4 of 112

About the workshop

Aim: Provide an overview of digital carbon estimation approaches and tooling, and help you take your first steps making calculations. Learn to use this to optimise Django / Python projects for carbon.

Duration: 50 mins

You will need:

A laptop (or share with someone)
A working internet connection
A modern browser
A Github account

4

GREEN WEB FOUNDATION

5 of 112

Learning objectives

5

How carbon emissions result from our use digital services, and why people are paying more attention
How they are measured and estimated, and common tools used to do so
How to use your existing Django knowledge to understand and manage carbon emissions from digital services

GREEN WEB FOUNDATION

6 of 112

Agenda

6

Housekeeping
About the Green Web Foundation
How people measure carbon. Why do it.
Making your first calculations
Using the Green Metrics Tool in a Django / Wagtail Project

GREEN WEB FOUNDATION

7 of 112

Accessing this deck

7

This deck is online:

https://www.thegreenwebfoundation.org/events/djangocon-eu-2024/

GREEN WEB FOUNDATION

8 of 112

Before we start properly

8

If there was one thing you would like to learn today about sustainable web development

what would it be (5 mins)��

bit.ly/djangocon-eu-2024-greening-digital-qn

GREEN WEB FOUNDATION

9 of 112

About the Green Web Foundation

GREEN WEB FOUNDATION

10 of 112

The Green Web Foundation is working towards a fossil-free internet by 2030.

10

“

GREEN WEB FOUNDATION

11 of 112

Build open source tools, APIs, and libraries

11

GREEN WEB FOUNDATION

12 of 112

Create prototypes.

Contribute to open source projects.

Publish open data.

12

GREEN WEB FOUNDATION

13 of 112

Talks.

Workshops.

Training.

Consulting.

13

GREEN WEB FOUNDATION

14 of 112

How digital causes emissions, and why measure it

GREEN WEB FOUNDATION

15 of 112

How do digital services cause emissions?

GREEN WEB FOUNDATION

15

Source: https://developers.thegreenwebfoundation.org/co2js/overview/

Datacentre usage

End user device usage

Network transfer

Production and disposal

Powering and cooling servers

Powering core networks, mobile networks,

on-premise wifi and fixed routers

Powering end user devices - laptops, workstations, smartphones and tablets

Energy usage from making all of the above - processing raw silicon and other raw materials into integrated circuits, batteries, metal casings, etc

GREEN WEB FOUNDATION

When we talk about digital services, what we might experience at our end might be playing a game, watching a video, or buying something on a device like a tablet or phone.

But in reality there’s a very complicated supply chain at work. When we use a service, we’re also relying on a network of always on undersea cables, routers towers and so on, which connect to a one more datacentres, which are also usually running 24 / 7. This all takes energy, which for the most part mostly comes from fossil fuels. All these things use energy when they’re turned on, i.e. when they’re used. So we call them use-phase emissions.

However someone also needs to actually make all these devices we hold, as well as the buildings that contain the servers and so on. This is often very energy intensive and water intensive, in it’s own right. You can think of us as putting energy into the creation of these things, so we usually called them embodied emissions

This might feel a bit clean and academic, so maybe it’s easier to understand if we have a look at some of the actual infrastructure we rely on.

16 of 112

How people estimate carbon emissions for digital

GREEN WEB FOUNDATION

17 of 112

All models are wrong, some models are useful.

GEORGE BOX

Empirical Model-Building and Response Surfaces, 1987

17

“

GREEN WEB FOUNDATION

18 of 112

Usage based

18

Billing based

Estimate emissions based on observed usage of one or more parts of a system.

Estimate emissions based on observed spend. Turn billing into emissions data.

GREEN WEB FOUNDATION

19 of 112

Usage based

19

Component view

System view

Global view

Billing based

“Native”

3rd party

GREEN WEB FOUNDATION

20 of 112

Usage based

20

Component view

System view

Global view

Instrument a single part of the system.��Self report own impact.

See impact of focussed changes.

Composable into bigger systems.

Instrument components in a system.

�Report at a system or service level.

Figure out where to focus efforts for things in production.

See impact of different configurations or usage scenarios

Use data from a single component to model your share of the global impact of digital

Very easy to start out with, but assumptions in models may not match your context.

GREEN WEB FOUNDATION

21 of 112

Usage based

21

Component view

System view

Global view

Firefox profiler:

Javascript, browser reports own energy / carbon use. �

Code Carbon:

Python, often used for AI, can wrap existing python code

Scaphandre:

Rust, gives server-side per process figures

Green Metrics Tool:

Python, simulates common usage scenarios

Greenframe

Proprietary, javascript. Simulates common usage scenarios

KEPLER

Kubernetes focussed, uses production telemetry data, growing corporate support

Website Carbon

Javascript, proprietary

Ecograder

Javascript, open source

Sitespeed.io

Javascript, used by Wikipedia, has own grafana dashboards

WebpageTest Carbon Control

php + js, available as hosted service

An incomplete list:

(All use sustainable web model under the hood))

GREEN WEB FOUNDATION

22 of 112

Billing based

22

Native

Third party

Consumes billing data about service, region, quantity.

Direct access to hardware.

Applies proprietary conversions to turn purchased capacity into carbon dioxide equivalents.

Each provider has a different, incompatible proprietary methodology.

Largely consumes same billing data as native.��Less access to hardware.

Uses external model to enrich billing data with environmental figures.

More consistent across cloud providers (if supported)

GREEN WEB FOUNDATION

23 of 112

Billing based

23

Native

Third party

Microsoft Emissions Impact Dashboard

Scopes 1, 2 and 3

AWS Customer Carbon Footprint Tool

Scopes 1, 2 (no scope 3)

Google Cloud Carbon Footprint

Scopes 1, 2 and 3

OVH

Scopes 1, 2 and 3, also water usage 👍

Cloud Carbon Footprint

Scopes 1,2 3 - first “significant” open source project

OpenCost

Recently merged PR now adds carbon costs to dashboards.

Green Pixie (proprietary)

Originally based on CCF, available as hosted service. Now offers richer data than just carbon.

(And more, each day)

An incomplete list:

GREEN WEB FOUNDATION

24 of 112

Usage based tools

24

Component view

System view

Global view

Firefox profiler

Javascript, reports measured browser energy

Code Carbon

Python, server side, often used for AI

Scaphandre

Rust, gives server-side per process figures

Green Metrics Tool

Python, simulates common usage scenarios

Greenframe

Proprietary, javascript simulates common usage scenarios

KEPLER

Kubernetes, uses production telemetry data for “real time” figures

Website Carbon

Javascript, proprietary

Ecograder

Javascript, open source

Sitespeed.io

Javascript, open source

Webpage Test Carbon Control

php + js, available as hosted service

(All use sustainable web design model under the hood))

GREEN WEB FOUNDATION

25 of 112

What these tools look like

GREEN WEB FOUNDATION

26 of 112

Who has used a website carbon calculator before?

GREEN WEB FOUNDATION

27 of 112

Website Carbon by Wholegrain Digital

Uses:

Sustainable Web Design with Green Web Dataset

27

GREEN WEB FOUNDATION

28 of 112

WebPageTest by Catchpoint

Uses:

Sustainable Web Design through CO2.js and Green Web Dataset

28

GREEN WEB FOUNDATION

29 of 112

WebPageTest Carbon Control

GREEN WEB FOUNDATION

30 of 112

Try it yourself:

WebPageTest.org

GREEN WEB FOUNDATION

31 of 112

Running a test

Simple Configuration

Enter a URL to test
Select a location
Check the Run Carbon Control option.
Start Test

31

GREEN WEB FOUNDATION

32 of 112

Running a test

Advanced Configuration

Enter a URL to test
Select a location & browser
Go to the Chromium tab

32

GREEN WEB FOUNDATION

33 of 112

Running a test

Advanced Configuration

4. Check the Run Carbon Control option.

5. Start Test!

33

GREEN WEB FOUNDATION

34 of 112

Reading test results

Find the

Carbon Control results section.

34

GREEN WEB FOUNDATION

35 of 112

Reading test results

Checking which domains used are hosted

green.

35

GREEN WEB FOUNDATION

36 of 112

Reading test results

Checking which domains used are hosted

green.

36

Click to expand!

GREEN WEB FOUNDATION

37 of 112

Reading test results

Estimated Carbon Footprint *

37

* For one page view. Assumes no caching.

GREEN WEB FOUNDATION

38 of 112

Reading test results

Best practices.

You can test these changes using WebPageTest’s Pro Opportunities & Experiments feature.

38

GREEN WEB FOUNDATION

39 of 112

Reading test results

Breakdown of resource types.

39

GREEN WEB FOUNDATION

40 of 112

For the nerds 🤓

Carbon Control results can be accessed via JSON.

40

GREEN WEB FOUNDATION

41 of 112

For the nerds 🤓

Carbon Control results can be accessed via JSON.

41

Search for the carbon-footprint key.

GREEN WEB FOUNDATION

42 of 112

WebPageTest Resources

42

GREEN WEB FOUNDATION

43 of 112

Here’s one I made earlier…

Don’t to want to run the test? Here’s the link to results

https://bit.ly/wpt-djangocon-eu-2024

This should point to:

https://www.webpagetest.org/result/240606_AiDcY4_F1V/1/carboncontrol/

43

GREEN WEB FOUNDATION

44 of 112

WebPageTest by Catchpoint

Uses:

Sustainable Web Design through CO2.js and Green Web Dataset

44

GREEN WEB FOUNDATION

45 of 112

Understanding how these work

Website carbon estimation tools are usage based, global view

45

Component view

System view

Global view

Billing based

“Native”

3rd party

Usage based

GREEN WEB FOUNDATION

46 of 112

Is just measuring transfer enough?

When deciding how we model impact we need be aware of the tradeoffs we are making when trying to come up with numbers we can use.

⚠️ This gets harder, very quickly, especially for a 50 minute workshop session. ⚠️

See this recent talk by my colleague Fershad Irani - The Nuance of Quantifying Digital Carbon Emissions

46

GREEN WEB FOUNDATION

47 of 112

Emerging Standards: Software Carbon Intensity

Think of “carbon emitted per activity”.

CO2e per page request

CO2e per CI run

CO2e per LLM query

Created by the Green Software Foundation - now a recognised ISO standard, with a freely available spec. Developed in the open on GitHub.

47

GREEN WEB FOUNDATION

48 of 112

48

GREEN WEB FOUNDATION

49 of 112

Green Coding AI from Green Coding Solutions GmbH

You measure more than just web pages. Does model choice make a difference with AI?

Can we use greener AI models instead?

49

GREEN WEB FOUNDATION

50 of 112

What questions green coding can help us answer

Asking a well known LLM about something we care about

50

GREEN WEB FOUNDATION

51 of 112

Optimising for carbon

Trying with a smaller model.

7x lower resource usage!

Was the other answer really 7x better?

51

GREEN WEB FOUNDATION

52 of 112

Green Metrics Tool Full stack measurements

GREEN WEB FOUNDATION

53 of 112

Green Metrics Tool from Green Coding Solutions GmbH

Imaginatively titled open source green metrics tooling. Python

53

GREEN WEB FOUNDATION

54 of 112

Green Metrics Tool (GMT) - what is it?

GMT is a usage based, system view tool, that simulates common interactions in a test system

54

Component view

System view

Global view

Billing based

“Native”

3rd party

Usage based

GREEN WEB FOUNDATION

55 of 112

How GMT works at very high level

Uses docker to set up a system, and record metrics from each container for various tasks

55

GREEN WEB FOUNDATION

56 of 112

What can you do with GMT?

Try different usage scenarios for your service - static vs dynamic, different page designs, etc.

56

https://www.green-coding.io/case-studies/wordpress-vs-hugo-cloudflare/

GREEN WEB FOUNDATION

57 of 112

Applying this to django - what you need

If you have a dockerised app, you’re halfway there to trying it out.

If you have any integration tests that drive a browser, you have a head start on usage scenarios too

57

Django app

Postgres

Browser

docker-compose.yaml

Redis

usage_scenario.yaml

+

scripts to drive the browser

GREEN WEB FOUNDATION

58 of 112

Learning from a real example - Wagtail and the bakery

Last year on the Wagtail GSoC project: used GMT to test common scenarios on the bakery demo site

58

GREEN WEB FOUNDATION

59 of 112

Learning from a real example - entire lifecycle

We can then measure the impact of running these common scenarios in the GMT’s dashboard.

SCI here is Software Carbon Intensity - CO2 per page request

59

GREEN WEB FOUNDATION

60 of 112

Learning from a real example - putting the SCI in CI

When run on a schedule, we can see how this changes over time, and aim to improve it.

60

GREEN WEB FOUNDATION

61 of 112

Trying Green Metric Tool ourselves

GREEN WEB FOUNDATION

62 of 112

What we’ll need to do

62

Fork a pre-prepared repo on github.
Set up your copy the Wagtail bakery demo, using docker compose.
Try out end to end tests, and see them in a usage_scenarios.yml file.
Notify hosted GMT service to run the usage scenarios in your repo on specialised hardware.
Bonus: Simulate a GMT server, to check your scenarios are working before submitting to the production service

GREEN WEB FOUNDATION

63 of 112

Trying it yourself - fork the pre-prepared repo

Organisation:�

thegreenwebfoundation��Project:�

bakery-gold-benchmark

Then follow the readme steps.

Docker required.

63

github.com/thegreenwebfoundation/bakerydemo-gold-benchmark

GREEN WEB FOUNDATION

64 of 112

Set up your copy the Wagtail bakery demo, using docker compose.

64

GREEN WEB FOUNDATION

65 of 112

See the end to end tests

65

GREEN WEB FOUNDATION

66 of 112

Run the puppeteer scenarios

You should see see them driving a full browser to interact with the rest of the system spun up with docker compose

66

GREEN WEB FOUNDATION

67 of 112

Notify the hosted GMT service to run the usage scenarios in your repo on specialised hardware.

67

GREEN WEB FOUNDATION

68 of 112

What we send to get notified

Your job is queued up - so add an email to send a notification to when results are ready

68

GREEN WEB FOUNDATION

69 of 112

What the notification looks like

Takes a few minutes.

69

GREEN WEB FOUNDATION

70 of 112

How to read the results

The steps we defined and ran in end-to-end scenarios should be visible.

70

GREEN WEB FOUNDATION

71 of 112

Seeing a software carbon intensity score

Our CO2 per page request score comes from an average of all the pages visited in all our scenarios.

71

GREEN WEB FOUNDATION

72 of 112

Seeing a software carbon intensity score over time

Wagtail’s bakery demo, taken every day over the last few weeks.

72

GREEN WEB FOUNDATION

73 of 112

Learning objectives (recap)

73

How carbon emissions result from our use digital services, and why people are paying more attention
How they are measured and estimated, and common tools used to do so
How to use your existing Django knowledge to understand and manage carbon emissions from digital services

GREEN WEB FOUNDATION

74 of 112

Thanks! I’ll be at the sprints if you want more

Please take 5 mins to provide the feedback form over lunch.

bit.ly/djangocon-eu-2024-greening-digital-feedback

chris@greenweb.org

www.linkedin.com/in/mrchrisadams

mastodon.social/@mrchrisadams�

climateAction.tech - a supportive climate-tech community

podcast.greensoftware.foundation - interviews of experts in this field�branch.climateAction.tech - online magazine about climate/tech�

74

GREEN WEB FOUNDATION

75 of 112

If you want to run GMT yourself

GREEN WEB FOUNDATION

76 of 112

You CAN self host Green Metrics Tool.

⚠️ You need hardware access, and time, and expertise for an accurate setup. ⚠️

BUT

You can run a version of GMT in the cloud if you just want to check file syntax, see containers are running as expected, and metrics are being collected.

It won’t have hardware access, so some figures will be modelled or look different.

76

GREEN WEB FOUNDATION

77 of 112

Trying it yourself in GitHub codespace

Visit green metrics tool repo on GitHub

You’ll set up a github codespace to run GMT.

You’ll then import your project for a measurement run.

77

GREEN WEB FOUNDATION

78 of 112

How to spin up a codespace

Click the drop down in the green code button
Choose codespaces
Choose "new with options”

78

GREEN WEB FOUNDATION

79 of 112

Choosing the correct codespace for this workshop

Choose “codespaces” branch
Choose “Workshop” configuration
The rest don’t matter so much for now.

79

GREEN WEB FOUNDATION

80 of 112

Follow steps into fetch a public repo

Let the codespace spin up
A file, splash.md opens up with helpful instructions
Copy and paste the steps into the codespace terminal to set it up for running measurement jobs

80

GREEN WEB FOUNDATION

81 of 112

Seeing metrics in your codespace

Click ports
Look at row labelled “metrics page”
Click the fiddly “world” icon in forwarded address
Click through the scary warning about visiting a running codespace

81

GREEN WEB FOUNDATION

82 of 112

What It should look like

You will have a running instance of GMT, and your project should be imported to peruse.

⚠️ Don’t forget to turn off the codespace at the end!

82

GREEN WEB FOUNDATION

83 of 112

Thanks! I’ll be at the sprints if you want more

Please take 5 mins to provide the feedback form over lunch.

bit.ly/djangocon-eu-2024-greening-digital-feedback

chris@greenweb.org

www.linkedin.com/in/mrchrisadams

mastodon.social/@mrchrisadams�

climateAction.tech - a supportive climate-tech community

podcast.greensoftware.foundation - interviews of experts in this field�branch.climateAction.tech - online magazine about climate/tech�

83

GREEN WEB FOUNDATION

84 of 112

Extra bonus bits

GREEN WEB FOUNDATION

85 of 112

Measuring in Firefox Profiler

GREEN WEB FOUNDATION

86 of 112

The best way to avoid ambiguity in carbon estimates is to measure the actual energy used.

GREEN WEB FOUNDATION

87 of 112

Capturing a profile

The following video shows how to capture a profile using the Firefox Browser.

Afterwards, we’ll walk through the steps of capturing and reading a profile.

87

GREEN WEB FOUNDATION

88 of 112

Capturing a profile

88

GREEN WEB FOUNDATION

89 of 112

Capturing a profile

Download the latest Firefox Browser

89

GREEN WEB FOUNDATION

90 of 112

Capturing a profile

Enable Firefox Profiler Menu Button

profiler.firefox.com

90

GREEN WEB FOUNDATION

91 of 112

Capturing a profile

Change the setting to Power.

91

GREEN WEB FOUNDATION

92 of 112

Power profiles will look different depending on the CPU you are running.

GREEN WEB FOUNDATION

93 of 112

Differing Power Profiles

Intel CPU: Captures the entire system. Includes background system tasks.
AMD CPU: Captures the entire system, but is broken down by CPU cores. Includes background system tasks.
Apple Silicon: Capture per process.

93

GREEN WEB FOUNDATION

94 of 112

Reading a profile

For the sake of time, we will use a pre-recorded profile.

https://share.firefox.dev/3QYQnCg

Profile captured on Apple M2 Mac.

94

GREEN WEB FOUNDATION

95 of 112

Reading a profile

Parent process

Core Firefox functionality (handling UI, navigation, profiles etc.)

95

GREEN WEB FOUNDATION

96 of 112

Reading a profile

Child process

Handles the content of the tab (loading content of the tab, including JavaScript)

96

GREEN WEB FOUNDATION

97 of 112

Reading a profile

View power & CO2e

Hover over the Power Process track.

97

GREEN WEB FOUNDATION

98 of 112

Reading a profile

View power & CO2e

Hover over the Power Process track.

98

Power consumption at that time(Watts)

Energy (milliwatt-hours) for the entire section

CO2e (milligrams)

GREEN WEB FOUNDATION

99 of 112

Reading a profile

CO2e Calculation

Energy used (mWh) x

GLOBAL Average Grid Intensity (g/kWh)

99

CO2e (milligrams)

GREEN WEB FOUNDATION

100 of 112

Reading a profile

Change the grid intensity

Open DevTools in your browser.
In the console type:

100

window.profile.meta.gramsOfCO2ePerKWh = 1000

GREEN WEB FOUNDATION

101 of 112

Reading a profile

Change the grid intensity

Now all CO2e calculations will use 1000g kWh

101

CO2e (milligrams)

GREEN WEB FOUNDATION

102 of 112

Firefox Profiler Resources

102

GREEN WEB FOUNDATION

103 of 112

An intro to CO2.js

GREEN WEB FOUNDATION

104 of 112

An open-source JavaScript library that enables developers to estimate the emissions related to use of their apps, websites, and software.

“

GREEN WEB FOUNDATION

105 of 112

Who’s using CO2.js today?

WebPageTest
Mozilla Firefox Profiler
Ecograder
Sitespeed
Datavizta (by Boavizta)
… and many more

105

GREEN WEB FOUNDATION

106 of 112

What can you do with CO2.js?

Estimate the carbon emissions of transferring bytes of data.
Convert energy usage data to carbon data - (access open grid intensity data by country)
Check if a website is hosted green using the Green Web Dataset.

106

GREEN WEB FOUNDATION

107 of 112

Data in CO2.js

Country-level grid intensity

Average annual data (from Ember)
Marginal annual data (from UNFCCC)

Green Web Dataset

Check a domain for green hosting

107

GREEN WEB FOUNDATION

108 of 112

Try it yourself:

CO2.JS Playground

https://observablehq.com/@greenweb/co2-js-playground

GREEN WEB FOUNDATION

109 of 112

How do they use it?

WebPageTest Carbon Control

Programming language: �PHP

Code: �GitHub

109

GREEN WEB FOUNDATION

110 of 112

How do they use it?

Mozilla Firefox Profiler

Programming language: �JavaScript

Code: �GitHub

110

GREEN WEB FOUNDATION

111 of 112

CO2.js Resources

111

GREEN WEB FOUNDATION

112 of 112

Want more? Drop us a line.

�Link to contact page.

112

GREEN WEB FOUNDATION