Denial of Service Vulnerabilities and Mitigation in O-RAN SMO Components for 5G Networks

The deployment of a Fifth-generation (5G) network worldwide has created a strong demand for architectures that emphasizes openness, interoperability, and flexibility. The Open Radio Access Network (O-RAN) initiative addresses this demand, but also introduces novel attack vectors. A central element of this architecture is the Service Management and Orchestration Framework (SMO) framework, responsible for lifecycle management, configuration, and assurance of distributed O-RAN components. This work investigates the susceptibility of O-RAN SMO components to Denial of Service (DoS) conditions by deploying the open-source O-RAN Software Community (OSC) SMO/Operations, Administration, and Maintenance (OAM) implementation in a controlled environment. Various DoS attacks targeting the O1 interface, more specifically the VNF Event Stream (VES) Collector and its associated services such as Kafka, are created and tested to analyze performance degradation, fault propagation, and resilience across the SMO stack. These tests demonstrate that overload of a single interface can cascade into broader system instability, impacting both event processing and management functions. Based on these experimental results, a new mitigation tool was developed in the form of a signature-based DoS detector. The final solution is dockerized, integrated into the OSC SMO/OAM implementation, and evaluated in the same test environment. The outcomes of this study highlight critical security gaps in the current SMO design and propose a practical defense mechanism to improve the robustness of O-RAN deployments against DoS attacks.