1 of 11

Grokking the grid:

(part of the The Green Web Foundation Open Syllabus project)

Creative Commons BY-SA 4.0

Just enough on electricity market design to help you see cloud differently

2 of 11

Grokking the grid

Just enough on electricity market design to help you see cloud differently

3 of 11

Nuclear

Coal

Fossil Gas

Marginal cost of energy

Cost of per KwH of generation

Total system load

Expensive up front., but sells volume, at low price into market.

Slow to ramp up and down.

Slighter faster to scale power output, but each extra unit of power more costly

Fast to scale up and down. Higher cost per unit generated.

Turned on at high demand

Where the power for digital infrastructure comes from

We can take some of these ideas to help us understand energy systems.

It’s worth knowing how energy markets work, as knowing this will give an idea what the cost structure of digital service looks like., and for big cloud providers, the second largest cost after their expensive staff is power.

The power we rely on for digital infrastructure is provided by grid operators, whose job it is to match supply and demand.

To do this, (and this is simplified) generators sell power, and the grid operator buys power from them, and then we end users buy power from the grid.

Because different kinds of generation have different properties, and because demand on the grid changes, the price of power changes. When there isn’t much load on the grid, nuclear, might sell into the grid first.

This is partly because it can generate power per unit at low price, but also because it has to, because it can’t easily scale up or down power, and in some cases it’s actually quite complicated and expensive to change this.

Coal power sells in at a higher rate, partly because coal is less energy dense than nuclear fuel, so you need more of it, increasing costs. However, ramping up or down power, while complicated, is cheaper than with nukes. For this reason, when there isn’t demand on the grid, and the price is low, coal can just switch off. It doesn't make money from selling power, but it doesn’t have to pay to burn coal either.

Gas by comparison is might use more expensive fuel to generate electricity, but because it can ramp supply up and down quickly, it can wait for the cost of energy to be high before selling into the grid, and not sell at all when the cost is low. And because it sells into the grid at a high price, it can still be profitable

4 of 11

Renewables

Nuclear

Coal

Fossil Gas

Marginal cost of energy

Cost of per KwH of generation

Total system load

Marginal cost is effectively zero

(no fuel to burn!)

Slow to ramp up and down. Sells volume, at low price into market

Slighter faster to scale power output, but each extra unit of power more costly

Fast to scale up and down. Higher cost per unit generated.

Turned on at high demand

Over the last 10 years we’ve seen the price of solar absolutely plummet, and as a result it’s affecting how grid markets work.

Because the fuel for renewables is free, they can sell in even cheaper than nukes, but they can only do so when it’s windy or sunny.

As you’d imagine, this doesn’t make people who own nukes happy, as now someone is undercutting them. But only when it’s sunny and windy.

This also means though that prices can be lower for power in times over abundance. In some cases, when there’s more supply on the grid than demand, the grid operator has a decision to make. They can reduce the price they’ll pay, which will result in some generators no longer selling into the grid, or in some extreme cases, they can ask operators to scale back production.

As I said before, it can be complicated to scale back generation for some generators, especially nukes. Grid operators might sometimes pay operators to scale back directly to account for the extra expense incurred by ramping back power, but another option is to just reduce the price so much that people will use more power, balancing things out by increasing demand, instead of reducing supply.

Sometimes demand can go so low that it goes negative - that is you can be paid to use power.

5 of 11

Renewables

Nuclear

Coal

Fossil Gas

Marginal cost of energy

Cost of per KwH of generation

Total system load

Storage

Loads up on power when cheap and green. Sells when power is expensive.

Marginal cost is effectively zero

(no fuel to burn!)

Battery storage can make this even worse for existing providers, because you can fill up batteries by buying when energy is cheap, or even negative because of renewables.

People who own storage can then sell back into the grid, and because their costs are lower, they can sell in at a price that’s lower than gas, and STILL make money.

I now tend to think of of renewables and storage like pincer movement from the future when it comes to energy markets - there’s SO much change going on at this later, and these are now squeezing dirtier technologies off the grid.

Further reading

https://www.benchmarkminerals.com/membership/china-is-building-one-battery-gigafactory-a-week-the-us-one-every-four-months-simon-moores/ - changes in battery storage

https://ieefa.org/wp-content/uploads/2019/06/Advances-in-Electricity-Storage-Suggest-Potential-Rapid-Disruption-of-U.S.-Electricity-Sector-1-1.pdf - analysis showing how renewable energy and battery will likely affect the sector

6 of 11

Most expensive per request. But you only ever pay per request, and the platform scales for you.

Bare metal

Virtual private server

Heroku, PaaS

Functions as a Service, etc.

Cost per unit of compute

Expensive up front, but if used all the time, cheaper in long run. Hard to scale up/down

More pricey for same performance, but easier to spin /up and down.

Pricier still, but you literally have a slider for scaling up/down.

Degree of abstraction

Let’s start with some ideas we might already be comfortable with.

Over the last ten years we’ve grown to abstract the compute resource from the underlying metal do a greater degree.

It’s useful to understand what that impact has on the cost per unit of compute though, and when you might use different kinds of compute.

For jobs you’re doing all the time, and you know you’ll do them for a long time, it might be cheaper to have hardware yourself, or depending on the scale you’re doing it at, use custom silicon - either in terms of a FPGA, or an ASIC.

Otherwise for jobs you’re using more, but ones that don’t justify the expense of going so custom, you might pay for more vanilla cloud, or take advantage of reserved instance pricing where load will be more predictable.

Finally, for work that is spikey, you might think of using on-demand services where you pay only for the time you use, like Amazon Lambda and so on, and accept that the costs of designing applications with this in mind are justified.

Further reading

https://datacentrereview.com/content-library/1632-the-smart-way-to-unlock-data-centre-power (on datacentres as active participants in energy and compute markets)

https://l.sdialliance.org/future-digital-utility-executive-summary - the sustainabla digital infrastrucure alliance on how to integrate datacentres into energy and heating grids

https://cacm.acm.org/magazines/2020/7/245702-a-domain-specific-supercomputer-for-training-deep-neural-networks/fulltext - Domain specific computing is an emerging technique for of compute carried out at a sufficient scale

https://cacm.acm.org/magazines/2020/7/245684-technical-perspective-asic-clouds-specializing-the-datacenter/fulltext -

https://cacm.acm.org/magazines/2020/7/245704-asic-clouds/fulltext - ASIC Clouds: Specializing the Datacenter for Planet-Scale Applications

7 of 11

“many workloads are spiky or drop to zero for extended periods… applications used in the workplace may only be active for 40 of the 168 hours in a week.”

Adrian Cockroft

@adrianco: AWS, ex Netflix

8 of 11

Renewables

Nuclear

Coal

Fossil Gas

Marginal cost of energy

Cost of per KwH of generation

Total system load

Marginal cost is effectively zero

(no fuel to burn!)

Slow to ramp up and down. Sells volume, at low price into market

Slighter faster to scale power output, but each extra unit of power more costly

Fast to scale up and down. Higher cost per unit generated.

Turned on at high demand

Over the last 10 years we’ve seen the price of solar absolutely plummet, and as a result it’s affecting how grid markets work.

Because the fuel for renewables is free, they can sell in even cheaper than nukes, but they can only do so when it’s windy or sunny.

As you’d imagine, this doesn’t make people who own nukes happy, as now someone is undercutting them. But only when it’s sunny and windy.

This also means though that prices can be lower for power in times over abundance. In some cases, when there’s more supply on the grid than demand, the grid operator has a decision to make. They can reduce the price they’ll pay, which will result in some generators no longer selling into the grid, or in some extreme cases, they can ask operators to scale back production.

As I said before, it can be complicated to scale back generation for some generators, especially nukes. Grid operators might sometimes pay operators to scale back directly to account for the extra expense incurred by ramping back power, but another option is to just reduce the price so much that people will use more power, balancing things out by increasing demand, instead of reducing supply.

Sometimes demand can go so low that it goes negative - that is you can be paid to use power.

9 of 11

Renewables

Nuclear

Coal

Fossil Gas

Marginal cost of energy

Cost of per KwH of generation

Total system load

Storage

Loads up on power when cheap and green. Sells when power is expensive.

Marginal cost is effectively zero

(no fuel to burn!)

Slow to ramp up and down. Sells volume, at low price into market

Slighter faster to scale power output, but each extra unit of power more costly

Fast to scale up and down. Higher cost per unit generated.

Turned on at high demand

Battery storage can make this even worse for existing providers, because you can fill up batteries by buying when energy is cheap, or even negative because of renewables.

People who own storage can then sell back into the grid, and because their costs are lower, they can sell in at a price that’s lower than gas, and STILL make money.

I now tend to think of of renewables and storage like pincer movement from the future when it comes to energy markets - there’s SO much change going on at this later, and these are now squeezing dirtier technologies off the grid.

Further reading

https://www.benchmarkminerals.com/membership/china-is-building-one-battery-gigafactory-a-week-the-us-one-every-four-months-simon-moores/ - changes in battery storage

https://ieefa.org/wp-content/uploads/2019/06/Advances-in-Electricity-Storage-Suggest-Potential-Rapid-Disruption-of-U.S.-Electricity-Sector-1-1.pdf - analysis showing how renewable energy and battery will likely affect the sector

10 of 11

Renewables

Nuclear

Coal

Fossil Gas

Cost of per KwH of generation

Total system load

As the total load increases, the cost of electricity increases too.

This means you don’t need to be running all the time to be viable if sell at a high enough price.

Battery storage can make this even worse for existing providers, because you can fill up batteries by buying when energy is cheap, or even negative because of renewables.

People who own storage can then sell back into the grid, and because their costs are lower, they can sell in at a price that’s lower than gas, and STILL make money.

I now tend to think of of renewables and storage like pincer movement from the future when it comes to energy markets - there’s SO much change going on at this later, and these are now squeezing dirtier technologies off the grid.

Further reading

https://www.benchmarkminerals.com/membership/china-is-building-one-battery-gigafactory-a-week-the-us-one-every-four-months-simon-moores/ - changes in battery storage

https://ieefa.org/wp-content/uploads/2019/06/Advances-in-Electricity-Storage-Suggest-Potential-Rapid-Disruption-of-U.S.-Electricity-Sector-1-1.pdf - analysis showing how renewable energy and battery will likely affect the sector

11 of 11

Bare metal

Virtual private server

Heroku, PaaS

Functions as a Service, etc.

Cost per unit of compute

Degree of abstraction

As the degree of abstraction from the hardware increases, so does the rough cost per unit of compute. But it’s often easier to ramp up and down.

Having more options lets you match

a workload to the appropriate kind

of compute, and make better use of

the hardware

Let’s start with some ideas we might already be comfortable with.

Over the last ten years we’ve grown to abstract the compute resource from the underlying metal do a greater degree.

It’s useful to understand what that impact has on the cost per unit of compute though, and when you might use different kinds of compute.

For jobs you’re doing all the time, and you know you’ll do them for a long time, it might be cheaper to have hardware yourself, or depending on the scale you’re doing it at, use custom silicon - either in terms of a FPGA, or an ASIC.

Otherwise for jobs you’re using more, but ones that don’t justify the expense of going so custom, you might pay for more vanilla cloud, or take advantage of reserved instance pricing where load will be more predictable.

Finally, for work that is spikey, you might think of using on-demand services where you pay only for the time you use, like Amazon Lambda and so on, and accept that the costs of designing applications with this in mind are justified.

Further reading

https://datacentrereview.com/content-library/1632-the-smart-way-to-unlock-data-centre-power (on datacentres as active participants in energy and compute markets)

https://l.sdialliance.org/future-digital-utility-executive-summary - the sustainabla digital infrastrucure alliance on how to integrate datacentres into energy and heating grids

https://cacm.acm.org/magazines/2020/7/245702-a-domain-specific-supercomputer-for-training-deep-neural-networks/fulltext - Domain specific computing is an emerging technique for of compute carried out at a sufficient scale

https://cacm.acm.org/magazines/2020/7/245684-technical-perspective-asic-clouds-specializing-the-datacenter/fulltext -

https://cacm.acm.org/magazines/2020/7/245704-asic-clouds/fulltext - ASIC Clouds: Specializing the Datacenter for Planet-Scale Applications