Lecture 11-12.�Q-learning ��

Sookyung Kim�

1

RECAP: Let’s see what that looks like

2

RECAP: Fixing the policy update

Remember

3

RECAP: Fixing the policy update

4

Fitted Value iteration

5

What if we don’t know the transition dynamics?

6

Can we do the “Max” Trick again?

(Cf)

7

Fitted Q-iteration

8

Review

9

From Q-iteration to Q-learning

• Why is this algorithm off-policy?

10

What is fitted Q-iteration optimizing?

11

Online Q-learning algorithms

12

Exploration with Q-learning

13

Exploration VS Exploitation

​

• Exploration vs Exploitation����
• Q-learning doesn’t have baked-in exploration in algorithm.

perform �the optimal/greedy action

random action

14

Exploration with Q-learning

15

Review

16