Metro-XHaul @IFIP/IEEE (IM 2019)

Metro-Haul has a conference contribution by Rafael Leira, Iván González, Jorge E. López de Vergara from Naudit;

Title: Wormhole streaming engine: A real 100 Gbit/s monitoring process using affordable Big

Network management is a never-ending challenge that increases in difficulty every year with the network evolution. This transformation affects directly the classical network monitoring process, which consists on very large dedicated and expensive monitoring equipment.
As an alternative, we have been working on a distributed monitoring system which allows monitoring not only different points along the network, but also at current link speeds, such as 100 Gbit/s. However, 100 Gbit/s monitoring represents a huge difference from the classical big data streaming problems, where even with high input rates the data is usually previously distributed and requires a complex computation, and therefore, a huge amount of computers.
It is not affordable nor adequate to build a large cluster to monitor a network. To solve this situation, we have created Wormhole, a streaming engine designed to take advantage of the latest network technologies and to grow mainly vertically but also horizontally. By mixing Wormhole as network monitoring entry point and other solutions like Apache Hadoop, Hive or Spot, we have designed a full system that is able to grow cheaply with the network and provides full analysis, from real-time statistics to machine learning predictions based on previous captures.