Supervision : Guy PUJOLLE

A monitoring and threat detection system using stream processing as a virtual function for Big Data

The late detection of security threats causes a significant increase in the risk of irreparable damages, disabling any defense attempt. As a consequence, fast real-time threat detection is mandatory for security guarantees.
In addition, Network Function Virtualization (NFV) provides new opportunities for efficient and low-cost security solutions. We propose a fast and efficient threat detection system based on stream processing and machine learning algorithms. The main contributions of this work are i) a novel monitoring threat detection system based on stream processing; ii) two datasets, first a dataset of synthetic security data containing both legitimate and malicious traffic, and the second, a week of real traffic of a telecommunications operator in Rio de Janeiro, Brazil; iii) a data pre-processing algorithm, a normalizing algorithm and an algorithm for fast feature selection based on the correlation between variables; iv) a virtualized network function in an open-source platform for providing a real-time threat detection service; v) near-optimal placement of sensors through a proposed heuristic for strategically positioning sensors in the network infrastructure, with a minimum number of sensors; and, finally, vi) a greedy algorithm that allocates on demand a sequence of virtual network functions.

2017-2022 Publications

• 2022
  • A. Lobato, M. ANDREONI LOPEZ, A. Cardenas, Otto Carlos M. B. Duarte, G. Pujolle : “A fast and accurate threat detection and prevention architecture using stream processing”, Concurrency and Computation: Practice and Experience, vol. 34 (3), pp. e6561, (Wiley) (2022)
• 2019
  • M. ANDREONI LOPEZ, D. Mattos, O. Carlos M. B. Duarte, G. Pujolle : “Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data”, Concurrency and Computation: Practice and Experience, (Wiley) (2019)
  • M. ANDREONI LOPEZ, D. Mattos, Otto Carlos M. B. Duarte, G. Pujolle : “A fast unsupervised preprocessing method for network monitoring”, Annals of Telecommunications - annales des télécommunications, vol. 74 (3-4), pp. 139-155, (Springer) (2019)
• 2018
  • M. Andreoni Lopez : “A monitoring and threat detection system using stream processing as a virtual function for Big Data”, thesis, phd defence 06/06/2018, supervision Pujolle, Guy (2018)
  • A. Lobato, M. ANDREONI LOPEZ, I. Sanz, A. Cardenas, Otto Carlos M. B. Duarte, G. Pujolle : “An Adaptive Real-Time Architecture for Zero-Day Threat Detection”, 2018 IEEE International Conference on Communications (ICC 2018), Kansas City, United States, pp. 1-6, (IEEE) (2018)
  • M. ANDREONI LOPEZ, A. Lobato, Otto Carlos M. B. Duarte, G. Pujolle : “An evaluation of a virtual network function for real-time threat detection using stream processing”, 2018 Fourth International Conference on Mobile and Secure Services (MobiSecServ), Miami Beach, United States, pp. 1-5, (IEEE) (2018)
• 2017
  • M. ANDREONI LOPEZ, R. Silva, I. Alvarenga, G. Rebello, I. Sanz, A. Lobato, D. Mattos, O. Duarte, G. Pujolle : “Collecting and characterizing a real broadband access network traffic dataset”, 2017 1^st Cyber Security in Networking Conference (CSNet), Rio de Janeiro, Brazil, pp. 1-8, (IEEE) (2017)
  • M. ANDREONI LOPEZ, A. Lobato, D. Mattos, I. Alvarenga, O. Duarte, G. Pujolle : “Um Algoritmo Não Supervisionado e Rápido para Seleção de Características em Classificação de Tráfego”, (2017)