Verifying Behavior of Reinforcement Learning Agents for Network Slice Admission Control
Jean Pierre Asdikian, Alaa Amro, Louma Mehyeddine, Carlos Natalino, Ihab Sbeity, Guido Maier, Paolo Monti, Sebastian Troia, Omran Ayoub
Politecnico di Milano, Milan, Italy,�Lebanese University, Lebanon,�Chalmers University of Technology, Gothenburg, Sweden�Scuola universitaria professionale della Svizzera italiana, Lugano, Switzerland�
RLAN 1st workshop, CNSM 2025, Bologna, Italy
Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico Di Milano
Network Slicing
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
2
NSR 1
Substrate Network
Overlay Network
(NSRs)
f
e
d
c
b
a
c
b
a
f
e
d
c
NSR 2
Network Slice Types
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
3
URLLC
Ultra-Reliable Low Latency Communications
mMTC
Massive Machine-Type Communications
eMBB
Enhanced Mobile Broadband
Autonomous
Driving
Industrial
Automation
Remote
Surgery
Mission Critical
Application
Intelligent
Transport
Smart City
Smart Home
VR/AR
Machine to Machine
Cloud Gaming
Cloud Office
GBps
data rate
3D/UHD
Video
HQ Voice
Admission Control in Network Slicing
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
4
A
B
E
F
C
D
NSR (t)
RL Agent
Substrate (t)
Verify Only
ILP
A
B
E
F
C
D
Substrate (t+Δt)
Rejected
Accepted
(Embed)
State
Request
Feedback
Reject
Accept
Output
Explainable AI in Deep Reinforcement Learning (DRL)
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
5
NSR 1
Substrate Network
Overlay Network
(NSRs)
f
e
d
c
b
a
c
b
a
f
e
d
c
NSR 2
Explainable AI in Deep Reinforcement Learning (DRL)
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
6
state s
Mask features (s-> s\C)
actor π(a|s)
𝑓(𝑠)=logit 𝜋(𝑎∣𝑠)
Re-evaluate actor
Shapley Values
Problem Statement
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
7
Methodology
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
8
RL Admission
Control Agent
Recorded States
States to Explain
SVERL
Obtain Explanations
Penalize Agent
Incoming NSR
Reward Agent
Substrate Network
ILP
Model
Send state
Verify
Disagree
Agree
Update
Network
Resources
Update reward
Update reward
1
2
3
4
5
6
7
8
After RL agent training
Embed slice if ILP and RL agent agree
Methodology
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
9
Methodology
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
10
Experimental Setup
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
11
Results
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
12
II
II
I
Results
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
13
III
IV
IV
II
II
I
Summary and Takeaways
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
14
Thank you
Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico Di Milano
Introduction: Reinforcement Learning Applications
Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>
16
Edge
Node
Edge
Node
Edge
Node
Edge
Node
Edge
Node
Edge
Node
Edge
Node
Edge
Node
Optical
Transport
Optical
Transport
Optical
Transport
Optical
Transport
Optical
Transport
Optical
Transport
Optical
Transport
Network Slice
Request 1 (NSR 1)
Network Slice
Request 2 (NSR 2)
Physical Network
Admission
Control
Optical
Networks
Edge
Caching
Origin
Server
Edge
Server
Edge
Server
Edge
Server
Edge
Server
Edge
Server
User
User
User
User
User
User
User
User
User
RL