The most natural approach to define Big Data is with the bigness. However according to Gartner, Big Data is defined as “high volume, high velocity and/or high variety information assets” that can be used to improve decision making and provide better insights.The majority of raw data, particularly Big Data, does not offer a lot of value in its unprocessed state. Big Data Analytic is the process of examining Big Data to uncover hidden patterns, unknown correlations and other useful information that can be used to make better decisions.
SDN paradigm to separate control plane and data plane has objective to create network abstraction for faster innovation. We can create new network applications without any need to interact directly with the actual network devices, instead our applications just need to consume the APIs provided by SDN controllers. SDN can be coupled with NFV, Network Function Virtualization, to run network functions and software on any open standard-based hardware to reduce CAPEX, OPEX, power and space. The orchestration of the network based on SDN and NFV can automate, provision, and interconnect all physical and virtual network resources.
Last year Gartner showed in their Hype Cycle that SDN & NFV is moving from hype to reality. The hype reached the peak on mid 2013 when most people thought that central controller using OpenFlow would control basic network devices completely, all network functions would transition to run on x86 platforms, and this paradigm would apply across to all aspects of networking. However, by mid 2015 most people understood that SDN & NFV implementation will use hybrid control plane consisting of a mix of distributed and centralized control plane, the network will consist of a mix of physical and virtual network functions, and SDN solutions are based on specific requirements / features and use-cases driven approach.
How can SDN & NFV be connected to Big Data & Analytic?
SDN applications must be responsive to the world in which they exist. If we are going to use SDN technologies to auto-provision network services, and turn the network to be truly dynamic, we will need to have feedback loops to ensure the desired behavior is actually occurring after a change is made. The natural progression from manual to automated will pass first through network analytics. As Michael Bushong wrote for Infoworld: "Ultimately, the promise of SDN will be inherently tied to the information that surrounds the network and drives the decisions that make SDN applications interesting. With more and more endpoints driving increasing traffic to a growing number of users, that adds up to big data."
Two weeks ago I was in Brussels to try Cisco WAN Automation Engine (WAE), a sample implementation of network analytic for Wide Area Network. To many network operators and service providers, WAN are oceans of uncertainty. Resource-constrained and multivendor, WANs produce delays and outages in far-flung and sometimes remote areas, posing a special set of issues that are distinct from those we see in data centers and access networks. WAN bandwidth is the most expensive bandwidth in the network and failure impacts are large.
WAE allows service providers to respond to overall traffic growth in terms of keeping the whole network stable and not just provisioning to traffic peaks. WAE offers what-if scenarios and failure analysis, empower the customer to be able to examine the impact of network failures in more detail, and was able to adopt a new policy by only upgrading links that were in danger of dropping packets on single-circuit failures. Using predictive analytics capabilities of WAE, we can build a powerful SDN solution that allows us to deploy and optimize innovative new services, such as global load balancing, bandwidth calendaring, bandwidth on demand, and premium network routing.
So we have seen how Big Data (Batman) and SDN (Superman) can work together, who should be the Wonder Woman in this story? It is obvious, isn't it? Internet of Things.
IoT and Big Data relationship forms Fog Computing, that uses one or a collaborative multitude of end-user clients or near-user edge devices to carry out a substantial amount of storage, communication and control, configuration, measurement and management. Fog computing can be perceived both in large cloud systems and big data structures, making reference to the growing difficulties in accessing information objectively.
With IoT it means everything is now connected. With huge explosion of number of endpoints, IoT will require a network that is not only self-provisioned, but as well as adaptive, dynamic and responsive that can be achieved with SDN. The network must provide broad and deep visibility into network traffic flow patterns too, the network must become the sensors, and this information can be used in different implementation such as for network security where by leveraging rich threat intelligence information allows for more rapid identification of security threats.
When the three superheroes are together, what should we call them?
Introducing AI @ Computer Networking. JP Vasseur, a Cisco Fellow, during Cisco Live Milan last year presented this as Self Learning Networks. Ex-Cisco CEO John Chambers mentioned this terminology in his keynote for Cisco Live 2014. Packet Pushers call this Machine Learning for Networks.
Artificial Intelligence, a singular consciousness that spawned an entire race of machines, as Morpheus explained to Neo in The Matrix back in 1999, is coming to unexpected places like computer networking. David Meyer, CTO & Chief Scientist at Brocade and formerly at Cisco, wrote the following notes: "it doesn’t take a lot of imagination to see how sophisticated unsupervised machine learning will revolutionize many of the more complex network tasks that are solved today in (much) less dynamic ways. For example, problems such as data center orchestration could benefit greatly from this technology. Clearly functionality that we’re talking about in networking these days, including Network Function Virtualization, Service Function Chaining, Mobility and the like are great candidates for treatment by Deep Learning. More generally, orchestration and optimization of Compute, Storage, Networking, Security and Energy (CSNSE) are prime candidates for consideration by Deep Learning technology. And consider what DevOps-style automation might look like when combined with Deep Learning."
Even IBM believes Watson, not SDN, is the future of networking. IBM's cognitive system demonstrates the power of artificial intelligence which will run our networks and transform IT management. They even come up with another cool name: AI-defined networks. And it won’t be about automating routine tasks, programmatically managing networks or providing a platform where humans create evermore clever access policies. Instead, we will teach machines that watch our network traffic, monitor our applications and cognitively recognize novel behavior on our firewalls.
So SDN is not the future of networking. And I agree. It is simply part of network evolution that is happening right now. If your network has not embraced SDN & NFV technologies, it is not a question of If but When. SDN will happen eventually.
The future of networking lies on the combination between SDN & NFV with Big Data & Analytics, and IoT. The three superheroes are working together and we may call them Self Learning Network, or Self Managing Network, or AI @ Computer Networking, or AI-defined Network.
Now let's just enjoy Batman vs. Superman movie.
