Security Evaluation of BGP in Modern Service Provider Networks: Challenges and Mitigations

Border Gateway Protocol (BGP) is the routing protocol that controls how independently operated networks, known as Autonomous Systems, exchange their reachability information with each other—essentially, BGP is the protocol that holds the Internet together, connecting tens of thousands of these networks into a single, globally reachable system. However, it was designed without any intrinsic mechanism to verify that a route announcement is authorized. This architectural weakness has led to real-world incidents, from the 2008 Pakistan Telecom hijack of YouTube’s address space to the 2018 Amazon Route 53 hijack that redirected users’ traffic to a phishing website. The Resource Public Key Infrastructure (RPKI), together with Route Origin Validation (ROV), is the primary deployed countermeasure, allowing a network to cryptographically verify that an announcing Autonomous System (AS) is authorized to originate a given prefix. However, RPKI-ROV validates only the origin of a route, not the AS_PATH the announcement claims to have traversed, leaving open the question of how much protection it actually provides against more capable adversaries. In this thesis, the effectiveness of RPKI-ROV against BGP prefix hijacking is evaluated, and the security boundaries of this mitigation are investigated through a controlled network emulation. A BGP topology was built using Containerlab, FRRouting, NLnet Labs’ Routinator, and ExaBGP. The route propagation was observed across six progressive scenarios, a clean baseline; an unmitigated sub-prefix hijack; the same hijack under RPKI-ROV enforcement, including a variant where a permissively-scoped ROA admits it as fully Valid; an AS_PATH forgery attack crafted to remain RPKI-valid while still diverting traffic; an enforcement-point boundary that re-exposes an otherwise-protected AS through an unfiltered session; and an interception attack that redirects and intercepts traffic in both directions, together with an exact-prefix variant of that attack that achieves the same bidirectional interception independently of ROA scope. The results confirm that RPKI-ROV can detect and block the sub-prefix hijack, which it was designed to address. However, the AS_PATH forgery scenario demonstrates that an attacker who forges the path rather than the origin can achieve a fully RPKI-valid hijack, with the RPKI validation state showing no indication of the forgery, particularly once the legitimate path becomes unavailable. The enforcement-boundary and interception scenarios show that these weaknesses are not independent, a permissively-scoped ROA and an unvalidated path can combine to turn an isolated redirection into a sustained, silent interception, and a separate combination of an unvalidated path with the enforcement-point boundary achieves the same bidirectional interception unconditionally, regardless of ROA scope, which indicates that no single mitigating configuration choice is sufficient to close this class of weakness on its own. These findings indicate that RPKI-ROV, as currently and typically deployed, provides meaningful but incomplete protection against BGP hijacking, and that path-level validation mechanisms such as BGPsec and Autonomous System Provider Authorization (ASPA) remain necessary to close the security boundaries this thesis identifies.