CFR-RL: Traffic Engineering with Reinforcement Learning in SDN

Junjie Zhang, Minghao Ye, Zehua Guo, Chen Yu Yen, H. Jonathan Chao

Research output: Contribution to journalArticle

Abstract

Traditional Traffic Engineering (TE) solutions can achieve the optimal or near-optimal performance by rerouting as many flows as possible. However, they do not usually consider the negative impact, such as packet out of order, when frequently rerouting flows in the network. To mitigate the impact of network disturbance, one promising TE solution is forwarding the majority of traffic flows using Equal-Cost Multi-Path (ECMP) and selectively rerouting a few critical flows using Software-Defined Networking (SDN) to balance link utilization of the network. However, critical flow rerouting is not trivial because the solution space for critical flow selection is enormous. Moreover, it is impossible to design a heuristic algorithm for this problem based on fixed and simple rules, since rule-based heuristics are unable to adapt to the changes of the traffic matrix and network dynamics. In this paper, we propose CFR-RL (Critical Flow Rerouting-Reinforcement Learning), a Reinforcement Learning-based scheme that learns a policy to select critical flows for each given traffic matrix automatically. CFR-RL then reroutes these selected critical flows to balance link utilization of the network by formulating and solving a simple Linear Programming (LP) problem. Extensive evaluations show that CFR-RL achieves near-optimal performance by rerouting only 10%-21.3% of total traffic.

Original languageEnglish (US)
Article number9109571
Pages (from-to)2249-2259
Number of pages11
JournalIEEE Journal on Selected Areas in Communications
Volume38
Issue number10
DOIs
StatePublished - Oct 2020

Keywords

  • Reinforcement learning
  • load balancing
  • network disturbance mitigation
  • software-defined networking
  • traffic engineering

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'CFR-RL: Traffic Engineering with Reinforcement Learning in SDN'. Together they form a unique fingerprint.

  • Cite this