Flux Research Group / School of Computing

Typhoon: An SDN Enhanced Real-Time Big Data Streaming Framework

Junguk Cho, Hyunseok Chang, Sarit Mukherjee, T.V. Lakshman, and Jacobus (Kobus) Van der Merwe

The 13th International Conference on emerging Networking EXperiments and Technologies (CoNEXT) 2017.

areas
Networking

abstract

Typhoon: An SDN Enhanced Real-Time Big Data Streaming Framework

Stream processing pipelines operated by current big data streaming frameworks present two problems. First, the pipelines are not exible, controllable, and programmable enough to accommodate dynamic streaming application needs. Second, the application-level data routing over the pipelines do not exhibit optimal performance for increasingly common one-to-many communication. To address these problems, we propose an SDN-based real-time big data streaming framework called Typhoon, that tightly integrates SDN functionality into a real-time stream framework. By partially offloading application-layer data routing and control to the network layer via SDN interfaces and protocols, Typhoon provides on-the- fly programmability of both the application and network layers, and achieve high-performance data routing. In addition, Typhoon SDN controller exposes cross-layer information, from both the application and the network, to SDN control plane applications to extend the framework’s functionality. We introduce several SDN control plane applications to illustrate these bene ts.