1 of 82

2 of 82

Lecture 01: Introduction & �Recap on Mathematical Optimization

2

Scan me:

Quizzes

​

Anonymous Questions (during or after the lecture)

vevox.app

ID:

148-282-057

The Lecture will be recorded

DTU Compute

2 February 2021

Welcome to 02435�Decision-making under uncertainty

3 of 82

Cheesy Introductory Statement

3

DTU Compute

Welcome to 02435�Decision-making under uncertainty

4 of 82

Cheesy Introductory Statement

4

What do all the following things have in common?

​

Climate change

Disease spreading

Global economy

Energy and Transportation systems

Logistics and supply chains

Your own health

Your relationships

Your grades

The taste of your coffee today

DTU Compute

Welcome to 02435�Decision-making under uncertainty

5 of 82

Cheesy Introductory Statement

5

What do all the following things have in common?

​

Climate change

Disease spreading

Global economy

Energy and Transportation systems

Logistics and supply chains

Your own health

Your relationships

Your grades

The taste of your coffee today

The fate of each is linked to good decision-making!

Actually, on a sequence of decisions.

…under uncertainty.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

6 of 82

Cheesy Introductory Statement

6

What do all the following things have in common?

​

Climate change

Disease spreading

Global economy

Energy and Transportation systems

Logistics and supply chains

Your own health

Your relationships

Your grades

The taste of your coffee today

The fate of each is linked to good decision-making!

Actually, on a sequence of decisions.

…under uncertainty.

…ah, and complexity.

In this course, we will learn about designing good decision-making algorithms.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

7 of 82

Course Scope

7

Decision 🡪 Information 🡪 Decision 🡪 Information 🡪 …

DTU Compute

Welcome to 02435�Decision-making under uncertainty

8 of 82

Course Scope

8

Decision 🡪 Information 🡪 Decision 🡪 Information 🡪 …

The fact that you get to act again in the future, �changes the way you act right now.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

9 of 82

Course Scope

9

Stochastic Optimization

Optimal Control

Model Predictive Control

Approximate Dynamic Programming

Reinforcement Learning

Decision 🡪 Information 🡪 Decision 🡪 Information 🡪 …

The fact that you get to act again in the future, �changes the way you act right now.

Simulation Optimization

DTU Compute

Welcome to 02435�Decision-making under uncertainty

10 of 82

Course Scope

10

Stochastic Optimization

…and (hopefully) make it to the other end wiser

Optimal Control

Model Predictive Control

Approximate Dynamic Programming

Reinforcement Learning

We will surf our way across these areas…

…borrowing concepts each time from where it seems more useful…

Decision 🡪 Information 🡪 Decision 🡪 Information 🡪 …

The fact that you get to act again in the future, �changes the way you act right now.

Simulation Optimization

DTU Compute

Welcome to 02435�Decision-making under uncertainty

11 of 82

Agenda for today

11

  1. Course Format

DTU Compute

Welcome to 02435�Decision-making under uncertainty

12 of 82

Agenda for today

12

  1. Course Format
  2. Lead Example

DTU Compute

Welcome to 02435�Decision-making under uncertainty

13 of 82

Agenda for today

13

  1. Course Format
  2. Lead Example
  3. General approach & Course Structure

Hour 1

DTU Compute

Welcome to 02435�Decision-making under uncertainty

14 of 82

Agenda for today

14

  1. Course Format
  2. Lead Example
  3. General approach & Course Structure
  4. Recap on Mathematical Optimization �

Hour 1

Hour 2

DTU Compute

Welcome to 02435�Decision-making under uncertainty

15 of 82

Agenda for today

15

  1. Course Format
  2. Lead Example
  3. General approach & Course Structure
  4. Recap on Mathematical Optimization �
  5. Presentation of the Assignment

Hour 1

Hour 2

Hours 3 and 4

DTU Compute

Welcome to 02435�Decision-making under uncertainty

16 of 82

Agenda for today

16

  1. Course Format
  2. Lead Example
  3. General approach & Course Structure
  4. Recap on Mathematical Optimization �
  5. Presentation of the Assignment
  6. Practice what you learned on the Lead Example

Hour 1

Hour 2

Hours 3 and 4

DTU Compute

Welcome to 02435�Decision-making under uncertainty

17 of 82

Course Format

17

Learning by Doing: No final exam. Only one big Assignment, given upfront.

The Assignment tries to simulate the situation where you are hired to design an algorithm for a real-world-like decision-making problem.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

18 of 82

Course Format

18

Learning by Doing: No final exam. Only one big Assignment, given upfront.

The Assignment tries to simulate the situation where you are hired to design an algorithm for a real-world-like decision-making problem.

Your solutions (codes) will be evaluated by simulating their performance in the “real” (read: simulated) system.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

19 of 82

Course Format

19

Learning by Doing: No final exam. Only one big Assignment, given upfront.

The Assignment tries to simulate the situation where you are hired to design an algorithm for a real-world-like decision-making problem.

Your solutions (codes) will be evaluated by simulating their performance in the “real” (read: simulated) system.

Assignment Task

Concept

Small Examples

Lead Example

Application to Assignment Task

DTU Compute

Welcome to 02435�Decision-making under uncertainty

20 of 82

Course Format

20

Learning by Doing: No final exam. Only one big Assignment, given upfront.

The Assignment tries to simulate the situation where you are hired to design an algorithm for a real-world-like decision-making problem.

Your solutions (codes) will be evaluated by simulating their performance in the “real” (read: simulated) system.

Us

You

Assignment Task

Concept

Small Examples

Lead Example

Application to Assignment Task

DTU Compute

Welcome to 02435�Decision-making under uncertainty

21 of 82

Lead Example: The Energy Hub

21

  • An energy hub with: wind turbine, electricity demand, electrolyzer, hydrogen storage, hydrogen to power, electricity grid connection.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

22 of 82

Lead Example: The Energy Hub

22

  • An energy hub with: wind turbine, electricity demand, electrolyzer, hydrogen storage, hydrogen to power, electricity grid connection.
  • For every hour that the electrolyzer is ON, it bears a fixed operational cost.
  • Wind power is free.
  • Grid prices vary over time.

​

"We need to operate this Energy Hub in a smart way i.e. make good decisions under uncertainty"

DTU Compute

Welcome to 02435�Decision-making under uncertainty

23 of 82

Lead Example: The Energy Hub

23

  • An energy hub with: wind turbine, electricity demand, electrolyzer, hydrogen storage, hydrogen to power, electricity grid connection.
  • For every hour that the electrolyzer is ON, it bears a fixed operational cost.

How would you approach this problem?

What would be the first thing you would do?

  • Wind power is free.
  • Grid prices vary over time.

​

"We need to operate this Energy Hub in a smart way i.e. make good decisions under uncertainty"

DTU Compute

Welcome to 02435�Decision-making under uncertainty

24 of 82

The process of designing �“Decision-making” frameworks

24

Define

Design

Evaluate

DTU Compute

Welcome to 02435�Decision-making under uncertainty

25 of 82

The process of designing �“Decision-making” frameworks

25

Define

Design

Evaluate

  1. What do we get to decide / control and how often?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Extracting this information from the stakeholder is often the biggest part of the job.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

26 of 82

Energy Hub Problem

26

  • An energy hub with: wind turbine, electricity demand, electrolyzer, hydrogen storage, hydrogen to power, electricity grid connection.
  • For every hour that the electrolyzer is ON, it bears a fixed operational cost.
  • Wind power is free.
  • Grid prices vary over time.

​

"We need to operate this Energy Hub in a smart way i.e. make good decisions under uncertainty"

Electrolyzer

H2�Storage

Fuel�Cell

Industry

Grid

Wind

  • The objective is to serve the power demand of the industry at minimum cost
  • The relevant decisions you need to make at each hour include: whether the electrolyzer is ON or OFF, the amount of power to draw from the grid, the amount of power to turn into hydrogen, and the amount of (previously stored) hydrogen to turn back into electricity.
  • At each hour we observe Storage Level, Wind Output and Grid Price. ​

​

DTU Compute

Welcome to 02435�Decision-making under uncertainty

27 of 82

The process of designing �“Decision-making” frameworks

27

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Markov Decision Process

DTU Compute

Welcome to 02435�Decision-making under uncertainty

28 of 82

The process of designing �“Decision-making” frameworks

28

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Begin with the simplest (dummy) strategy.

e.g. electrolyzer always OFF & just use your wind power and draw the rest from the grid.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

29 of 82

The process of designing �“Decision-making” frameworks

29

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Begin with the simplest (dummy) strategy.

Often leads to revisiting the problem definition.�

DTU Compute

Welcome to 02435�Decision-making under uncertainty

30 of 82

The process of designing �“Decision-making” frameworks

30

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Begin with the simplest (dummy) strategy.

�How do you know how good/bad it is?

DTU Compute

Welcome to 02435�Decision-making under uncertainty

31 of 82

The process of designing �“Decision-making” frameworks

31

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Begin with the simplest (dummy) strategy.

�How do you know how good/bad it is?

Build a simulation environment on which any strategy can be evaluated.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

32 of 82

The process of designing �“Decision-making” frameworks

32

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Begin with the simplest (dummy) strategy.

​

​

​

Build a simulation environment on which any strategy can be evaluated.

Now we design algorithms.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

33 of 82

The process of designing �“Decision-making” frameworks

33

Define

Design

Evaluate

  1. What do we get to decide / control?
  2. What is our performance metric?
  3. What are the sources of uncertainty?
  4. How exactly are these connected?

Begin with the simplest (dummy) strategy.

​

​

​

Build a simulation environment on which any strategy can be evaluated.

The evaluation environment is related to the problem definition and independent of the designed strategy

DTU Compute

Welcome to 02435�Decision-making under uncertainty

34 of 82

The Method is not The Problem Definition

34

DTU Compute

Welcome to 02435�Decision-making under uncertainty

35 of 82

The Method is not The Problem Definition

35

“So, your leg hurts huh?

Classic ice-and-rest problem…”

“Let’s apply ice and rest!”

DTU Compute

Welcome to 02435�Decision-making under uncertainty

36 of 82

Always

36

DTU Compute

Welcome to 02435�Decision-making under uncertainty

37 of 82

Define

37

DTU Compute

Welcome to 02435�Decision-making under uncertainty

38 of 82

The Problem

38

DTU Compute

Welcome to 02435�Decision-making under uncertainty

39 of 82

First.

39

DTU Compute

Welcome to 02435�Decision-making under uncertainty

40 of 82

Course Plan

40

DTU Compute

Welcome to 02435�Decision-making under uncertainty

41 of 82

Course Plan

41

+ Code a Simulation Environment

DTU Compute

Welcome to 02435�Decision-making under uncertainty

42 of 82

Course Plan

42

+ Code a Simulation Environment

DTU Compute

Welcome to 02435�Decision-making under uncertainty

43 of 82

Course Plan

43

+ Code a Simulation Environment

DTU Compute

Welcome to 02435�Decision-making under uncertainty

44 of 82

Course Plan

44

+ Code a Simulation Environment

DTU Compute

Welcome to 02435�Decision-making under uncertainty

45 of 82

Course Plan

45

DTU Compute

Welcome to 02435�Decision-making under uncertainty

46 of 82

Course Plan

46

without�Uncertainty

DTU Compute

Welcome to 02435�Decision-making under uncertainty

47 of 82

Course Plan

without�Uncertainty

DTU Compute

Welcome to 02435�Decision-making under uncertainty

48 of 82

Resources

  1. Mykel J. Kochenderfer, Tim A. Wheeler and Kyle H. Wray, Algorithms for Decision Making
  2. Shapiro, A., Dentcheva, D., & Ruszczynski, A., Lectures on stochastic programming: modeling and theory.
  3. Birge, J. R., & Louveaux, F. Introduction to stochastic programming.
  4. Warren Powell, Sequential Decision Analytics and Modeling: Modeling with Python
  5. Warren Powell, Reinforcement Learning and Stochastic Optimization
  6. Sutton & Barto, Reinforcement Learning
  7. Dimitri Bertsekas, Dynamic Programming and Optimal Control

DTU Compute

Welcome to 02435�Decision-making under uncertainty

49 of 82

Recap on Mathematical Optimization

49

DTU Compute

2 February 2021

Welcome to 02435�Decision-making under uncertainty

50 of 82

Let’s model your current decision-making problem: How much effort to invest in this course

50

  1. What do you get to decide / control?
  2. What do you care about?
  3. What are the sources of uncertainty?

Write some keywords �(decisions, objectives, uncertain factors)

vevox.app

ID:

148-282-057

DTU Compute

Welcome to 02435�Decision-making under uncertainty

51 of 82

Let’s model your another student’s decision-making problem: How much effort to invest in this course

51

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

52 of 82

Let’s model your another student’s decision-making problem: How much effort to invest in this course

 

52

 

This is a LP

DTU Compute

Welcome to 02435�Decision-making under uncertainty

53 of 82

Mathematical Optimization

53

DTU Compute

Welcome to 02435�Decision-making under uncertainty

54 of 82

Types of Optimization Problems

54

DTU Compute

Welcome to 02435�Decision-making under uncertainty

55 of 82

Elements of an Optimization Problem

55

DTU Compute

Welcome to 02435�Decision-making under uncertainty

56 of 82

Elements of an Optimization Problem

56

DTU Compute

Welcome to 02435�Decision-making under uncertainty

57 of 82

Elements of an Optimization Problem

57

DTU Compute

Welcome to 02435�Decision-making under uncertainty

58 of 82

Linear Programming (LP)

58

DTU Compute

Welcome to 02435�Decision-making under uncertainty

59 of 82

Feasible and Optimal Solutions

59

DTU Compute

Welcome to 02435�Decision-making under uncertainty

60 of 82

Possible Outcomes

60

  1. Optimal Solution�
  2. Infeasible Problem�
  3. Unbounded Problem

DTU Compute

Welcome to 02435�Decision-making under uncertainty

61 of 82

Possible Outcomes

61

  1. Optimal Solution
  2. Infeasible Problem
  3. Unbounded Problem

​

​

 

Perform a sanity check for this problem

vevox.app

ID:

148-282-057

DTU Compute

Welcome to 02435�Decision-making under uncertainty

62 of 82

Possible Outcomes

62

  1. Optimal Solution
  2. Infeasible Problem
  3. Unbounded Problem

​

​

 

And now for this problem

vevox.app

ID:

148-282-057

DTU Compute

Welcome to 02435�Decision-making under uncertainty

63 of 82

The student’s problem

 

63

 

but indices still italic

DTU Compute

Welcome to 02435�Decision-making under uncertainty

64 of 82

The student’s problem extended

 

64

 

This is a MILP

DTU Compute

Welcome to 02435�Decision-making under uncertainty

65 of 82

The student’s problem extended

 

65

 

This is a MILP

Is the optimal solution of this problem better or worse than the one of the previous student’s problem?

vevox.app ID: 148-282-057

DTU Compute

Welcome to 02435�Decision-making under uncertainty

66 of 82

The student’s problem extended further

 

66

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

67 of 82

67

 

 

67

The student’s problem extended further

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

68 of 82

68

 

 

68

The student’s problem extended further

 

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

69 of 82

69

 

 

69

The student’s problem extended further

If stressed, need rest

 

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

70 of 82

70

 

 

70

 

The student’s problem extended further

 

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

71 of 82

71

 

 

71

The student’s problem extended further

 

 

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

72 of 82

 

72

 

72

The student’s problem extended further

 

 

 

Here, M=1 is a good choice (can you see why?)

DTU Compute

Welcome to 02435�Decision-making under uncertainty

73 of 82

73

 

73

The student’s problem: StudentProblem.py

 

 

 

 

DTU Compute

Welcome to 02435�Decision-making under uncertainty

74 of 82

Pyomo Code

74

 

You can play with StudentProblem.py – available on Learn

DTU Compute

Welcome to 02435�Decision-making under uncertainty

75 of 82

Pyomo Code

75

Why did I include 15 weeks instead of 13?

What if you want a grade 95, 75, or 45?

How does Lecture Quality (0.3 instead of 0.35) affect things?

You can play with StudentProblem.py – available on Learn

DTU Compute

Welcome to 02435�Decision-making under uncertainty

76 of 82

Questions & Group Enrollment

76

vevox.app ID: 148-282-057

  1. Write any questions (anonymously) on Vevox:

​

2. Go on Learn and enroll in a Group.

​

3. Contact your group and sit together for the Exercise session.

​

DTU Compute

Welcome to 02435�Decision-making under uncertainty

77 of 82

Assignment

77

DTU Compute

2 February 2021

Welcome to 02435�Decision-making under uncertainty

78 of 82

Assignment

78

DTU Compute

Welcome to 02435�Decision-making under uncertainty

79 of 82

Assignment

79

  • Groups for the Assignment are open: You must enroll yourself in a group via DTU Learn in order to participate (and see the Assignment).
  • Groups are up to 4 members each. There will be no automatic enrollment.
  • You can enroll in a group until February 14th; but it is strongly recommended that you enroll asap.
  • Deadline for Tasks 1 & 2: March 2nd , 23.59
  • If you are already in a group and looking for additional members, you can use the Discussions section to let your colleagues know.
  • If you are looking for a group to enroll, prioritize groups that have open requests in the Discussions.
  • Once enrolled with a group, contact your group through DTU Learn or via email right away.

​

  • You can start working on Task 1, from 16.30 today.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

80 of 82

On LLMs

80

Exercises

​

Use them and also use the TAs.

​

LLMs are very helpful if you know what you are doing.

If you don’t, they can “chaos” you.

​

​

Assignment

​

Do not ask TAs.

​

Use of LLMs is up to you.

​

“The LLM said so” is not a valid justification for something you did.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

81 of 82

What’s next

81

Us

You

Assignment Task

Concept

Small Examples

Lead Example

Application to Assignment Task

It is tempting to skip the Exercise and jump directly to the Assignment. �Don’t.

DTU Compute

Welcome to 02435�Decision-making under uncertainty

82 of 82

What to do now

82

A) Go to

Content 🡪 Exercise - Lead Example

and do Exercise 1.

​

B) After 16.30:

Go to

Assignment

Get together with your Group and start working on the Assignment – Task 1

DTU Compute

Welcome to 02435�Decision-making under uncertainty