Supervision : Pierre SENS

Co-supervision : BOUBENDIR Amina, GUILLEMIN Fabrice

Optimization of Network Slice Placement in Distributed Large Scale Infrastructures : From Heuristics to Controlled Deep Reinforcement Learning

Network Slicing is a major stake in 5G networks and beyond, which is mainly enabled by Network Function Virtualization (NFV) and Software Defined Networks (SDN). These two paradigms enable telcos to offer virtual networks meeting specific needs of the vertical markets on the top on the same shared physical infrastructure. An important challenge in the implementation at scale of Network Slicing is Network Slice Placement and the optimal allocation of virtualized resources. This can be formulated as a multi-objective Integer Linear Programming (ILP) problem. However, ILP suffers from combinatorial explosion when solving NP-hard problem, especially for large-scale scenarios. Heuristics and Machine Learning (ML) algorithms have been investigated as efficient means of tackling this kind of problem. This PhD thesis investigates how to optimize Network Slice Placement in distributed large-scale infrastructures focusing on online heuristic and Deep Reinforcement Learning (DRL) based approaches. First, we rely on ILP to propose a data model for enabling on-Edge and on-Network Slice Placement. In contrary to most studies related to placement in the NFV context, the proposed ILP model considers complex Network Slice topologies and pays special attention to the geographic location of Network Slice Users and its impact on the End-to-End (E2E) latency. Extensive numerical experiments show the relevance of taking into account the user location constraints. Then, we rely on an approach called the ”Power of Two Choices”(P2C) to propose an online heuristic algorithm for the problem which is adapted to support placement on large-scale distributed infrastructures while integrating Edge-specific constraints. The evaluation results show the good performance of the heuristic that solves the problem in few seconds under a large-scale scenario. The heuristic also improves the acceptance ratio of Network Slice Placement Requests when compared against a deterministic online ILP-based solution. Finally, we investigate the use of ML methods, more specifically DRL, for increasing scalability and automation of Network Slice Placement considering a multi-objective optimization approach to the problem. We first propose a DRL algorithm for Network Slice Placement which relies on the Advantage Actor Critic algorithm for fast learning, and Graph Convolutional Networks for feature extraction automation. Then, we propose an approach we call Heuristically-Assisted Deep Reinforcement Learning (HA-DRL), which uses heuristics to control the learning and execution of the DRL agent. We evaluate this solution through simulations under stationary, cycle-stationary and non-stationary network load conditions. The evaluation results show that the heuristic control is an efficient way of speeding up the learning process of DRL, achieving a substantial gain in resource utilization, reducing performance degradation, and is more reliable under unpredictable changes in network load than non-controlled DRL algorithms.

2020-2022 Publications

• 2022
  • J. Alves Esteves, A. Boubendir, F. GUILLEMIN, P. Sens : “On the Robustness of Controlled Deep Reinforcement Learning for Slice Placement”, Journal of Network and Systems Management, vol. 30 (3), pp. 43, (Springer Verlag) (2022)
• 2021
  • J. Alves Esteves : “Optimization of Network Slice Placement in Distributed Large Scale Infrastructures : From Heuristics to Controlled Deep Reinforcement Learning”, thesis, phd defence 12/13/2021, supervision Sens, Pierre, co-supervision : Boubendir, Amina, Guillemin, Fabrice (2021)
  • J. Alves Esteves, A. Boubendir, F. GUILLEMIN, P. Sens : “A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement”, IEEE Transactions on Network and Service Management, (IEEE) (2021)
  • J. Alves Esteves, A. Boubendir, F. GUILLEMIN, P. Sens : “DRL-based Slice Placement Under Non-Stationary Conditions”, CNSM 2021 - 17^th International Conference on Network and Service Management, Izmir, Turkey (2021)
  • J. Alves Esteves, A. Boubendir, F. Guillemin, P. Sens : “DRL-based Slice Placement under Realistic Network Load Conditions”, CNSM 2021 - 17^th International Conference on Network and Service Management, Izmir, Turkey (2021)
• 2020
  • J. Alves Esteves, A. Boubendir, F. Guillemin, P. Sens : “Heuristic for Edge-enabled Network Slicing Optimization using the "Power of Two Choices"”, CNSM 2020 - 16^th International Conference on Network and Service Management, Izmir / Virtual, Turkey (2020)
  • J. Alves Esteves, A. Boubendir, F. Guillemin, P. Sens : “Edge-enabled Optimized Network Slicing in Large Scale Networks”, NoF 2020 - 11^th International Conference on Network of the Future, Bordeaux / Virtual, France (2020)
  • J. Alves Esteves, A. Boubendir, F. Guillemin, P. Sens : “Location-based Data Model for Optimized Network Slice Placement”, NetSoft 2020 - 6^th IEEE International Conference on Network Softwarization, Ghent / Virtual, Belgium, pp. 404-412, (IEEE) (2020)
  • J. Alves Esteves, A. Boubendir, F. Guillemin, P. Sens : “Optimized Network Slicing Proof-of-Concept with Interactive Gaming Use Case”, ICIN 2020 - 23^rd Conference on Innovation in Clouds, Internet and Networks and Workshops, Paris, France, pp. 150-152, (IEEE) (2020)