1 of 16

Verifying Behavior of Reinforcement Learning Agents for Network Slice Admission Control

Jean Pierre Asdikian, Alaa Amro, Louma Mehyeddine, Carlos Natalino, Ihab Sbeity, Guido Maier, Paolo Monti, Sebastian Troia, Omran Ayoub

Politecnico di Milano, Milan, Italy,�Lebanese University, Lebanon,�Chalmers University of Technology, Gothenburg, Sweden�Scuola universitaria professionale della Svizzera italiana, Lugano, Switzerland�

RLAN 1st workshop, CNSM 2025, Bologna, Italy

Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico Di Milano

2 of 16

Network Slicing

  • What it is: A partition of computational and network resources

​

​

  • Function: Allocate resources to nodes and links for a specific service

​

​

  • Feature: Multiple slices can co-exist on the same substrate network

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

2

NSR 1

Substrate Network

Overlay Network

(NSRs)

f

e

d

c

b

a

c

b

a

f

e

d

c

NSR 2

3 of 16

Network Slice Types

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

3

URLLC

Ultra-Reliable Low Latency Communications

mMTC

Massive Machine-Type Communications

eMBB

Enhanced Mobile Broadband

Autonomous

Driving

Industrial

Automation

Remote

Surgery

Mission Critical

Application

Intelligent

Transport

Smart City

Smart Home

VR/AR

Machine to Machine

Cloud Gaming

Cloud Office

GBps

data rate

3D/UHD

Video

HQ Voice

4 of 16

Admission Control in Network Slicing

  • Role: It either accepts or rejects a slice request on the network

​

  • Function: Given the NSR’s demands and the current SN state, decide on Accept/Reject via RL agent. On Accept, the Integer Linear Programming (ILP) attempts the embedding and updates the SN, on Reject the ILP runs a feasibility-only check and returns feedback.

​

  • ILP objective: Minimize the usage of CPU/BW of the substrate

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

4

A

B

E

F

C

D

NSR (t)

RL Agent

Substrate (t)

Verify Only

ILP

A

B

E

F

C

D

Substrate (t+Δt)

Rejected

Accepted

(Embed)

State

Request

Feedback

Reject

Accept

Output

5 of 16

Explainable AI in Deep Reinforcement Learning (DRL)

  • Black-box issue: RL policies can accept/reject a Network Slice Request (NSR) without explanations, operators can’t trust or approve deployment
  • What is needed: Action-level reasons on the visited states to verify/validate the learned policy before deployment
  • Ground truth for feasibility: Use ILP to benchmark feasibility and faithful reward assignment during embedding
  • Focus: Explain proactive rejections, counter-intuitive cases where the agent rejects a NSR when it should’ve been accepted

​

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

5

NSR 1

Substrate Network

Overlay Network

(NSRs)

f

e

d

c

b

a

c

b

a

f

e

d

c

NSR 2

6 of 16

Explainable AI in Deep Reinforcement Learning (DRL)

  •  

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

6

state s

Mask features (s-> s\C)

actor π(a|s)

𝑓(𝑠)=logit 𝜋(𝑎∣𝑠)

Re-evaluate actor

Shapley Values

7 of 16

Problem Statement

  • Problem Statement: RL admission control sometimes rejects NSRs that the ILP could embed. Without reasons, operators cannot verify/validate the policy before deployment.

​

  • Objective: Use action-level explanations on visited states via SVERL to attribute the policy admit/reject score to concrete substrate/request features and assess policy behavior

​

  • Research Questions:
    • RQ1: Which features most influence the admit/reject decisions of the RL agent?
    • RQ2: Why do proactive rejection cases occur and what substrate conditions drive them?
    • RQ3: Can these explanations verify/validate the policy (e.g., reveal conservative bias under congestion)?

​

​

​

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

7

8 of 16

Methodology

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

8

​

RL Admission

Control Agent

Recorded States

States to Explain

SVERL

Obtain Explanations

Penalize Agent

Incoming NSR

Reward Agent

Substrate Network

ILP

Model

Send state

Verify

Disagree

Agree

Update

Network

Resources

Update reward

Update reward

1

2

3

4

5

6

7

8

After RL agent training

Embed slice if ILP and RL agent agree

9 of 16

Methodology

  •  

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

9

10 of 16

Methodology

  • What to explain: The policy’s score for that action (logits), computed by the actor on each visited state
  • Record states after training: run the trained agent, save states and a snapshot of the substrate so we can reset the environment consistently
  • Mask features: Hide a subset of features in the state and fill them with typical values taken from our recorded states
  • Re-evaluate the actor: Run the policy again on the masked state and see how the score changes
  • Aggregate with Shapley: Combine many such tests to get per-feature contributions

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

10

11 of 16

Experimental Setup

  • Environment:
    • Substrate: 15 nodes
    • NSRs per episode: 50
    • Node CPU capacity: 50 – 80 units
    • Link BW capacity: 30 – 50 units
    • Slice type: URLLC

​

​

​

  • Training and Optimization:
    • Number of episodes: 300
    • Learning rate: 1e-3
    • Discount factor: 0.85
    • Cosine Annealing: 0.01

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

11

  • Reward system:
    • Agree + Feasible: + revenue/cost ∈ (0,1]
    • Agree + Infeasible: -0.4
    • Reject + Feasible: 0
    • Reject + Infeasible: +0.4

​

12 of 16

Results

  • RL agent and ILP in agreement: 71% of the time (whether to accept or reject)
  • Proactive rejection cases: 29%
  • No case of agent accepting embedding when there were not enough resources�
  • In Inset I, free CPU and BW have low feature values whilst occupied CPU/BW have high ones
  • The agent is more prone to reject if either resource is highly congested
  • This behavior is backed by Inset II, as the number of nodes and links embedded reach high capacity usage

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

12

II

II

I

13 of 16

Results

  • In Inset III, the features only have a minor effect on the decision

​

  • Inset IV shows that request size and path-length features have only minor effects on the decision

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

13

III

IV

IV

II

II

I

14 of 16

Summary and Takeaways

  • Summary:
    • Problem: RL admission control is a black box. We need reasons, especially for proactive rejections
    • Approach: A2C+LSTM AC agent + ILP + SVERL for action-level explanations on the states the policy actually visits.
    • What we explain: The action score and each feature’s rank and direction (pushes accept vs reject).
    • Evaluation protocol: Train the agent. At test time ILP embeds only on Accept, and checks feasibility
    • Explanation extraction (SVERL): Record visited states after training, mask subsets of features with typical values from those states, re-evaluate the actor, aggregate with Shapley to get per-feature contributions.

​

  • Key Takeaways:
    • Outcomes: Capacity respected. Agent behavior is conservative
    • Drivers of Proactive Rejection: CPU/BW saturation and high embedded load dominate. Topology (avg/std degree) and request size/path length matter less.
    • Operator benefit: Explanations verify/validate the policy and show when rejections stem from congestion rather than request demand.

​

​

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

14

15 of 16

Thank you

Jean Pierre Asdikian

DEIB, Edificio 20, Politecnico di Milano

jeanpierre.asdiguian@polimi.it

Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico Di Milano

16 of 16

Introduction: Reinforcement Learning Applications

Jean Pierre Asdikian <jeanpierre.asdiguian@polimi.it>

16

Edge

Node

Edge

Node

Edge

Node

Edge

Node

Edge

Node

Edge

Node

Edge

Node

Edge

Node

Optical

Transport

Optical

Transport

Optical

Transport

Optical

Transport

Optical

Transport

Optical

Transport

Optical

Transport

Network Slice

Request 1 (NSR 1)

Network Slice

Request 2 (NSR 2)

Physical Network

Admission

Control

Optical

Networks

Edge

Caching

Origin

Server

Edge

Server

Edge

Server

Edge

Server

Edge

Server

Edge

Server

User

User

User

User

User

User

User

User

User

RL