Typhoon: An SDN Enhanced Real-Time Big Data Streaming Framework
The 13th International Conference on emerging Networking EXperiments and Technologies (CoNEXT) 2017
abstract
Stream processing pipelines operated by current big data streaming frameworks
present two problems. First, the pipelines are not flexible, controllable, and
programmable enough to accommodate dynamic streaming application needs. Second,
the application-level data routing over the pipelines do not exhibit optimal
performance for increasingly common one-to-many communication. To address
these problems, we propose an SDN-based real-time big data streaming framework
called Typhoon, that tightly integrates SDN functionality into a real-time
stream framework. By partially offloading application-layer data routing and
control to the network layer via SDN interfaces and protocols, Typhoon provides
on-the-fly programmability of both the application and network layers, and
achieve high-performance data routing. In addition, Typhoon SDN controller
exposes cross-layer information, from both the application and the network,
to SDN control plane applications to extend the framework’s functionality.
We introduce several SDN control plane applications to illustrate these
benefits.
areas
Networking