1 of 8

Addressing Signal Delay in Deep Reinforcement Learning

ICLR 2024 Spotlight

Wei Wang Dongqi Han Xufang Luo Dongsheng Li

​

A brief overview, without going into details.

Grasp the main idea in just 5 min.

2 of 8

A Short summary

This paper investigates signal delay issues within the scope of reinforcement learning, presenting both theoretical solutions and practical implementations using the Actor-Critic framework.

3 of 8

Why is this important?

​

How do we solve it? Theory analysis + Practical solution

​

What? Experiment result

Content of This video

4 of 8

Why?

Because Signal Delay is Common and Significant

5 of 8

  • All Signal delay is equivalent from the agent view
  • Action history helps to recover Markovian Property

Contributions - Theoretical Findings

6 of 8

Delay-Reconciled Training

for Critic

State Augmentation

for Actor

Contributions - A general framework for Actor-Critic

  • Add Action history
  • Asymmetric Actor-Critic
  • Exploring: Predicting as supervision

“Action history helps to recover Markovian Property”

7 of 8

Main Results

Significantly improvements

8 of 8

Addressing Signal Delay in Deep Reinforcement Learning

Wei Wang Dongqi Han Xufang Luo Dongsheng Li

A brief overview, without going into details.

Thanks!