CFR RL: Traffic Engineering With Reinforcement Learning in SDN
Traditional Traffic Engineering (TE) solutions can achieve the optimal or near optimal performance by rerouting as many flows as possible. However, they do not usually consider the negative impact, such as packet out of order, when frequently rerouting flows in the network. To mitigate the impact of network disturbance, one promising TE solution is forwarding the majority of traffic flows using Equal Cost Multi Path (ECMP) and selectively rerouting a few critical flows using Software Defined Networking (SDN) to balance link utilization of the network. However, critical flow rerouting is not trivial because the solution space for critical flow selection is enormous. Moreover, it is impossible to design a heuristic algorithm for this problem based on fixed and simple rules, since rule based heuristics are unable to adapt to the changes of the traffic matrix and network dynamics. In this paper, we propose CFR RL (Critical Flow Rerouting Reinforcement Learning) a Reinforcement Learning based scheme that learns a policy to select critical flows for each given traffic matrix automatically. CFR RL then reroutes these selected critical flows to balance link utilization of the network by formulating and solving a simple Linear Programming (LP) problem. Extensive evaluations show that CFR RL achieves near optimal performance by rerouting only 10% 21.3% of total traffic.
IEEE Journal on Selected Areas in Communications