SDN Warriors

SDN Warriors Facebook Group is an open group for any Network Engineer who wants to transform to become SDN & NFV Architect, Network Programmability Engineer and Network DevOps. The group is run by mentors who are currently transforming ourselves and willing to help others to do the same. We have the skills and experiences with various SDN solutions, we have done SDN & NFV projects, we have hosted SDN Hackathon event, we have even created our own SDN products.
Non-SDN related topics will be banned, and please use English only
https://www.facebook.com/groups/sdnwarriors/

How to Bring SDN/NFV into Reality

Unless you've been living inside a cave, or on top of a mountain without any Internet connection, you must have heard or read the news about Software-Defined Networking (SDN). In fact, SDN news pops up too often these days it makes some skeptics start thinking whether it is really real or just another hype in networking industry.

The challenge is it seems like everybody comes with their own definition of SDN. Each networking vendor displays its solution based on each own interpretation of SDN implementation. IETF group called the Interface to the Routing System (I2RS) is still trying to standardize southbound programming protocols and network-wide, multilayer topologies that include both virtual and real elements, network overlays and underlays. Open Networking Foundation (ONF), as a user-driven organization dedicated to the promotion and adoption of SDN, until today is mainly focusing on standardization of OpenFlow protocol. And the rise of new SDN startups, no doubt have created lots of excitement with many innovations within SDN spaces, contributes to the confusion at the same time.

The questions from today's business leaders in companies that consume networking technologies: if we want to embrace SDN, are we on the right track? Which way to go? For some, the question is even more basic: What to do to get to SDN? How to start?

To try to find the answers to those questions, perhaps we should refresh our memory how it was all started. Everybody has heard the story how some smart people at Stanford University initiated Project Clean Slate, to explore what kind of Internet we would design if we were to start with a clean slate and 20-30 years of hindsight. Project Clean Slate led the development of OpenFlow.

According to Wikipedia, OpenFlow is a Layer 2 communication protocol that gives access to the forwarding plane of a network switch or router over the network. In "traditional" network paradigm, control and data (forwarding) plane reside within the physical device such as router or switch. In SDN paradigm, not all processing happens inside the same device. Control plane is separated from the physical device that will run only as forwarding plane.

OpenFlow has four parts:

Controller - resides on a server and provides control plane function for the network
OpenFlow Agent - resides on a network devices and fulfill requests from the Controller
Northbound APIs - enable applications to interface with the Controller
OpenFlow Protocol - the Layer 2 protocol that the Controller and Agents use to communicate

Open Networking Foundation (ONF) was formed to accelerate the delivery and use of SDN by standardizing OpenFlow. Deutsche Telekom, Facebook, Goldman Sachs, Yahoo, Google, Microsoft, NTT, Verizon are the board members, while almost all known vendors in networking industry like Cisco, HP, Juniper, Intel etc. sit as ONF members. That shows how important OpenFlow is.

However, the important point to keep in mind that OpenFlow does not equal to SDN. OpenFlow is just one protocol within SDN paradigm. Cisco once shared sample of new protocols for SDN along with their place in multiple layers: from device API layer, to forwarding, control plane, network services, orchestration and management.

Now, you have read this far and may have the One Most Important Question in mind: who cares? What should I care with all this? Who cares about new protocols in the network? How can it help me with my business? Exactly.

Let's first quickly review who were the first ones making noise about SDN in real network: James Hamilton from Amazon released a famous white paper in 2010 saying "Datacenter Networks Are In My Way". As network engineers we have to admit that auto provisioning technique in networking space is way behind the server space. Automation and orchestration in provisioning new Virtual Machine in Data Center can take only minutes to build, to configure the system and even to install service packages (e.g. Apache, DNS etc) in order to make it ready to provide services to the users. While to provision the new VLANs and subnets, to inject the new prefix into Routing Protocol, to create or re-configure Security policy and QoS... well, most networks today are still using manual provisioning method that require at least days or even weeks to complete those tasks.

Urs Holzle, Senior Vice President of Technology Infrastructure at Google, gave keynote speech at the
second annual Open Networking Summit (April 2012) and mentioned that Google has been using OpenFlow as the protocol for their WAN. That was a pretty big statement and somewhat validated OpenFlow as a viable technology in the SDN space.

You may argue that you are not Amazon nor Google. You don't run Massive Scale Data Center that requires multi-tenancy and very fast auto provisioning for services. And you are still happy with your current WAN since the traditional networking protocols that you use can provide predictable behavior, "self healing" the network will find alternate path when the primary path is down, and your network can provide sub-second convergence time during failure. You don't want to even consider new protocol, or new paradigm, that is not proven yet to replace the current one you've been using for years.

And you are absolutely right. For us mere mortals, nothing has changed in networking since Cisco built their first router about 30 years ago. The "traditional" network paradigm has remained mostly intact, until mid-2012. That was the time when VMWare bought a startup company called Nicira, that created Network Virtualization Platform as overlay network, for around $1.2 billion in total. Rumor has it that Cisco was trying to acquire Nicira too, but outbid by VMWare at that time.

What does it mean?

When major networking players such as VMWare and Cisco are willing to spend more than a billion dollar to acquire a startup in SDN, that's a market validation. That gives a strong signal to confirm that SDN business is real. And obviously that acquisition was able to boost lots of investors to started investing in many other SDN startups.

Software defined networking (SDN), as per Wikipedia, is an approach to computer networking that allows network administrators to manage network services through abstraction of lower-level functionality. To understand this abstraction concept, let's see the different types of Network Programmability.

The traditional network device we know, shown as the most left in the picture, can be managed by the applications using well known protocol from device CLI to SNMP or Netflow. Picture (1) shows how several network vendors have created vendor specific API to access their devices to allow some degree of programmability by the applications, even though we have to follow what is defined in the API. When SDN with OpenFlow comes to the picture (2a) it separates the control plane and data/forwarding plane. This "Pure SDN" approach lets the controller to provide Northbound API for the applications, and to use Southbound API to communicate with network device. Northbound API can be open/standard based or vendor specific, and the Southbound API can be OpenFlow, PCEP, I2RS and other types of open API, or vendor specific API. The result is an abstraction where the application does not need to deal directly with lower level functionality within network device.

Some people argue that all networking technologies that have been built for the past three decades can't be thrown away and replaced by new SDN protocols. Hence they say it's better to keep the network as it is, but have the controller ready to program the network when it's required. "Preserve what works, program when needed". This is the idea behind Hybrid SDN (2b) approach, where the network can continue with its predictable behavior, self healing, and fast convergence, but can be re-program based on the input from the applications. Just like pure SDN, controller can sit between applications and network device to provide the abstraction.

What is overlay networks (3)? When we have physical network built using physical devices and links, called the underlay, then on top of it we create new virtual network topology that can be different with the physical network, then we get an overlay network. It is not a new concept in networking. GRE tunnels or MPLS VPN are some example of overlay network we've been using for years. That idea is being sold by the vendors who sell overlay networks as SDN solution: leave the underlay network as it is, and build completely new virtual network on top of it.

Before we start looking for the answer of which network programmability is the best, let's discuss NFV quickly. Network Functions Virtualization (NFV) is a network architecture concept that proposes using IT virtualization related technologies to virtualize entire classes of network node functions into building blocks that may be connected, or chained, to create communication services. So instead of having dedicated hardware for router, switch, firewall, cache engine, DPI, BRAS, NAT etc. we can run those network functions as software on top of common x86 hardware.

That's not a new idea either, is it? Before network devices use dedicated hardware with either "merchant silicon" off-the-shelf chip or vendor's designed-specific ASIC, we used to have Linux firewall or Linux router. So we used to have network functions running as software on x86. What's new? NFV brings the idea to not only virtualized the network functions but to connect them as well, known as "Service Chaining", in order to provide benefits to the business such as time reduction for service provisioning, and dynamic and elastic scale for network services.

Obviously not all network functions make sense to be virtualized. For Network Forwarding function with high throughput, high bandwidth, stateless and predictable, using designated hardware or custom Network Processing Unit (NPU) is a better way. But for many Network Services function with low to medium throughput, stateful and unpredictable, this is the area where virtualization on x86 can shine. In short, when you need more muscle functions, it's better to use special designed hardware. When you need more brain functions, it's better to virtualized the functions.

Now we reach the $1 million question:
How to bring SDN, NFV, Overlay, Programmability etc into reality?

Based on my experience working in this space the past few years, to make any SDN/NFV solutions successful in real world:

1. It must solve Real business problem

Different customers have different business problem to solve. Separation of control plane and data plane as offered by OpenFlow may not be interesting to others outside research or academic institution.

For Massive Scaled Data Center or SP Data Center or Cloud provider, what matter is the automation of new services to scale the multi-tenancy environment. Deep programmability or network overlay may be interesting to support application mobility. For Service Providers that have invested heavily on the WAN infrastructure, what important is to have deep network programmability to monetize the current infrastructure. WAN optimization is another requirement, as well as providing SLAs to their customers. For Enterprise customers, especially the ones that only use network infrastructure to support their core business, the requirement is to have auto provisioning and device config management to provide services.

2. It should be orchestrated to deliver business outcome

SDN to separate the control plane and data plane, and NFV to have network functions and software running on any open standards-based hardware, mean nothing if they don't deliver business outcome. A new component called Services Orchestration should be used to provide automation, provisioning and interworking of physical and virtual resources in order to enable new service offering.

The industry agrees that the combination of these capabilities offer business a path to agility and dynamic networks. To optimally benefit from virtualization, the intersection of all three must be addressed – and on a carrier grade level which means services can be scaled up as well as down easily with complete reliability and security. The confluence of SDN, NFV, and orchestration takes advantage of cloud economics and provides service automation so we can nimbly address new market opportunities and dynamically modify existing services to best benefit the business. With this approach service innovation dramatically changes from weeks and months to minutes and days.

3. It has to follow industry standard and open architecture reference

We don't have industry standard yet for SDN architecture reference. However, ETSI - European Telecommunications Standards Institute has already published NFV Reference Architectural Framework as shown below.

The three main components of the reference architecture:

- VNF is the actual Network Function application, for example virtual router, virtual switch, virtual firewall, virtual load balancer, virtual web filter and so on, along with the Element Management System (EMS)
- NVF Infrastructure consists of virtualization layer that provides abstraction between the physical resources (compute, storage, network hardware) and virtual resources, where we can run the VNFs
- NVF MANO includes Virtualized Infrastructure Manager as resource manager to manage NFVI, VNF Manager to manage the life cycle of the VNFs, and the Orchestrator manager to orchestrate the overall solution

ETSI even defines formalized NFV use cases with potentially virtualized functions.

4. It should be modular, loosely coupled and extensible

Any SDN/NFV solutions will be built using various components. Modular architecture provides benefits since it can be scaled and extended when it's needed. One important note here is: every component should be connected as loosely coupled, and not "locked". It means the solution is expected to run in multi-vendor environment and will have lots of integration to build the whole system.

The picture above shows an example of SDN solution architecture. All physical and virtual resources are managed by Management and Orchestration box, that consists of network control, compute and storage control and service control, with cross-domain orchestration to manage all of them. Customer facing service portal is common to provide self service to the users. Management and Orchestration box provides northbound API for the application, and use various southbound API from all the control layer to the resources. Many other boxes like billing system, trouble management, inventory and service assurance can be attached to have the end-to-end solution.

5. It has to use Open API

In order to make sure all components can interconnect without any vendor locked-in, the solution must use Open API between each component. The Northbound API between Management/Orchestration and Applications, for example, can use REST API, NetConf, SOAP or even XML RPC with standard data format like XML and JSON. The Southbound API from Management/Orchestration toward compute, network and storage resources can use NetConf, OpenFlow, SNMP, Telnet/SSH or even device specific API as long as it is open.

6. It must have continuous development and support, and embrace FOSS

There are many components from SDN solution that can embrace Free and Open Source Software (FOSS). The operating system for the Compute resource, for example, can use Linux. Virtual Infrastructure Manager can use OpenStack. Network controller can use OpenDaylight. All programming languages used like Python or Java are free to use. The main benefit to embrace FOSS components in the solution is we can get continuous development and support from the community, and we don't need to create all the components in the solution only some focused components that we think matter the most.

Do you think there is SDN/NFV solution today following the points above?

I can give you one example of SDN/NFV solution that complies with all the six points above. And you may be surprised when I tell you that kind of solution comes from a network vendor that is known to be closed source: Cisco. Yes, Cisco with its Virtualized Managed Services (VMS) actually follow all the points in the list as explained below:

1. VMS solves SP's real business problem in managed services

SPs that offer VPN solution have seen the strong adoption by large and medium enterprises, however the aggregate growth rate is pretty much stable (around 5-7% in many countries). This requires them to find the blue ocean, a new market with expansion opportunity such as Small and Medium Business. However, SMB is very price sensitive and has adopted public cloud services to shift the workload. Huge number of SMBs provides indeed big opportunity, however the solution must be attractive from pricing perspective, time to market/provision the service, and integrated or bundled with other service offering.

2. VMS is orchestrated to cut down time to provision

VMS uses the idea of cloud-based managed service by moving the network functions from CPE to SP cloud.

The CPE is moving to be less and less intelligent, to a point where it becomes L2 network insertion device only. Virtualized Network Functions in SP Cloud are service chained to provide new service offering to the customer. All initiation and configuration of the VNFs and CPE are done automatically to provide agile service provisioning to the end customer. The CPE even uses zero touch deployment mechanism to reduce the requirements to have a technician sent to the location only to fix the device. Once the CPE is powered up, it will call home to register itself to the system and start building encrypted tunnel to its assigned service chained VNFs.

3. VMS follows ETSI architecture reference

I can only provide the following diagram to show Cisco VMS architecture.

However, I can mention that Cisco VMS follows ETSI architecture reference:

- Cisco has extensive list of VNFs from virtual router to virtual web filter, and there is nothing can stop the solution to use third party VNFs
- NVF Infrastructure uses Linux KVM as virtualization layer, running on top of physical hardware resources for compute, network and storage. Again, even though the solution is built using Cisco UCS and networking products, it can accommodate third party hardware as well
- NVF MANO includes OpenStack as Virtualized Infrastructure Manager, Cisco ESC as VNF life cycle manager, and Cisco NSO (previously known as Tail-f NCS) as the Cross Domain Orchestrator

4. VMS is modular, loosely coupled between components, extensible

VMS solution uses modular architecture. There is no locked-in between components. The solution can be integrated with customer existing setup. It can also be extended beyond the main functionality when needed.

5. Yes, all APIs between VMS components are open

The self-service portal can send the service request to orchestrator using Netconf. NSO as orchestrator can use NetConf or REST API to instruct Cisco ESC to initiate the VNFs, which requires ESC to tell OpenStack either with REST or Nova Client API to spin up the VMs. Cisco NSO can provision the configuration of network devices either using NetConf, SSH, REST or device specific API.

6. VMS embraces FOSS

As you can see above, Cisco VMS embraces FOSS like OpenStack, Linux, KVM along with open API to connect them to Cisco specific components. By doing that, Cisco can focus on the component that matters the most like the orchestrator. And innovation with life cycle manager or end-to-end system integration is the competitive advantage from Cisco compare to the other vendors.

Before you accuse me as Cisco fanboy, let me make a statement first: I'm not trying to sell Cisco solution here. Cisco has the best sales force in the market to do that. I just want to give example that even a market leader like Cisco is willing to adapt; to change from closed source to be more open as shown in its VMS solution. This is a big shift in networking industry. And I believe we will see more and more solutions that are inline with the six points above.

And if you are in a company that consumes networking technology, next time any networking vendor comes to offer their SDN solution, you can ask them questions if the solution offered really solves your business problem, how the solution is orchestrated to deliver business outcome, whether the solution follow industry standard and open architecture reference, ask if the solution is modular so can be extended easily without any locked-in to specific vendor, check if it supports open API, and if they have plan to embrace FOSS to ensure continuous development and support.

That, I believe, is the only way to bring SDN/NFV into reality.

Message from New York to All CCIEs

Skill Development Planning by the Dozen

I’ve been going to Cisco Live as one of the speakers for several years now. As Cisco employee, you can go to Cisco Live (for free) only if you are a speaker, or part of the World of Solutions exhibition, or if you come for the customers, or to support the network infrastructure for the event.

For the past three years at Cisco Live I’ve been sharing strategy and tips and tricks of how to become CCIE based on my own experience. My part of the session is focusing on skill development planning to achieve the objective, which is to pass CCIE lab exam. There are many technical sessions available during the event, but only very few talk about how to build learning plan and walk you through step by step of sample plan created by someone who has done it. I believe the session material can be applied outside CCIE context, and it’s relevant with my previous post, so I’m going to share it here with some updates.

Robert Grant mentioned that strategy is the means by which individuals achieve their objectives. In short, successful strategy can be achieved by having clear and consistent goal, understanding the environment or the industry as well as our capabilities, and executing with effective structure and system.

You must have a strategy for your skill development. And here are a dozen steps you can follow to build one:

1. Begin with End in Mind

What do you see at the end of the road? What is your objective? What are you planning to accomplish? Having a clear goal can help you to plan on how to achieve it. Your goal can be your one thing. Whatever it is, you must have a goal or objective. If you don’t have it yet, go find one.

To make it easier to digest, I will use an example here. Let’s say today you are a network engineer who puts a goal to become Network Function Virtualization (NFV) Solutions Architect someday.

2. Break It Down

How to eat an elephant? Cut it into pieces and eat one piece at a time. Big goal must be dismantled into pieces so we can understand it well.

What is NFV? What does Solutions Architect do?

As per Wikipedia, NFV is a network architecture concept that proposes using IT virtualization related technologies to virtualize entire classes of network node functions into building blocks that may be connected, or chained, to create communication services. As described by ETSI NFV whitepaper, Virtualization eliminates the dependency between a network function (NF) and its hardware, as seen in typical physical network appliances, by creating a standardized execution environment and management interfaces for the Virtualized Network Functions (VNFs). ETSI defined NFV framework into three main components: Virtualized Network Function (VNF), NFV Infrastructure, and NFV Management and Orchestration (MANO).

If we break down each NFV component, we can see sample of a product or solution that we have to learn:

- VNF is the actual Network Function application, could be run as Virtual Machine (VM). Sample of this are virtual router, virtual switch, virtual firewall, virtual load balancer, virtual web filter and so on
- NVF Infrastructure consists of virtualization layer, that can be server hypervisor such as VMware ESXi or Linux KVM, and physical hardware resources for compute, network and storage. This includes the server host operating system (e.g. Linux), physical server, networking devices and underlying protocols (routing, switching), and storage disk, SAN as well as protocols such as iSCSI, NFS and FCoE
- NVF MANO includes Virtualized Infrastructure Manager like OpenStack, VMware, SDN controller such as Open Daylight, VNF Life cycle manager and Cross Domain Orchestrator such as Tail-f NCS

Don’t forget the northbound/southbound and interface to connect between the components: Rest API, OpenFlow, NetConf, data format with Json and XML, and if you are into data modeling: YANG. And some of the components require customization and module expansion using programming language such as Python and Java.

Solutions Architect is someone who can articulate problem statements, solution capabilities and the gaps between the two. He or she must correlate business issues around technology and processes, and able to create an architectural solution specific to a given customer or segment. Solutions Architect needs to interpret business requirements and strategies, and mapping them to solution capabilities, and driving the integration of the solution into an existing infrastructure, which would drive systems integration and implementation approaches.

Solutions Architect skill requirements go beyond the end-to-end technical architecture:

- deep expertise in several technology domains (network/infrastructure, data, applications/middleware)
- fluency in network technologies and related advanced technologies
- expertise in several soundness disciplines (availability, security) as well as operational disciplines (systems management, change management)
- fluency in key IT business architecture areas including business process design and governance

To become Solutions Architect one must possess customer skill, business oriented mindset, communication, ability to collaborate, planning skill, leadership, and problem solving skill. Above all, it is important for a Solutions Architect to be able to solve the puzzle, put all the pieces together, and to connect the dots.

3. Self Assessment

Now you have a better picture of your goal, it is important to understand where you are standing today and see the gaps. Bruno, CCIE program manager and my co-speaker at Cisco Live, is very famous with his mantra: "Know what you don’t know".

If you know what you don’t know, then you can build the plan to acquire the knowledge. My advise is to build some kind like checklist, let’s say listing both the NFV components and the Solution Architect skillset from the above point. Then make self assessment about your current situation with scale between 1 to 5 with level 1 as the lowest (no knowledge at all) and level 5 as the highest (posses the expertise to design, implement and advise others, being the “go-to guy”):

- how good are you with linux OS? (1-5)
- how good are you with hypervisor e.g. ESXi, KVM? (1-5)
- how good are you with OpenStack? (1-5)
- how fluent are you in Java? (1-5)
And so on.

You can do the same with non-technical skills:

- how is your communication skill? (1-5)
- how is your ability to articulate solution? (1-5)
- how good is your problem solving skill? (1-5)
- how good is your understanding of customer business? (1-5)
And so on.

4. Define Target Performance Level

This is the question I always ask from time to time: How do you define success?
If you have the list of required skills to become NFV Solutions Architect from both technical and non-technical, and you have done self assessment to understand your current situation, what level should you target for each skill?

In a perfect world, we need to be 5 in everything. But we don’t live in a perfect world.

The challenge will be the time. Our time is limited so we have to prioritize. We need to define which skills we have to develop to reach level 5 and which skills we can develop only to reach level 2 or 3, at least for time being.

To pass CCIE Routing & Switching lab, for example, you must possess level 5 of core knowledge such as IGP Routing, BGP, MPLS. They are defined as core knowledge due to the amount of exam questions about the topic, and in the lab that technology is the foundation that put everything together. You can’t build a working solution if you are unable to make the core technology works. You may reach only level 3 for QoS or security, and still able to pass the exam since they are not the core knowledge.

But how we define target performance level for NFV Solutions Architect?
My advise is to start from the top. NFV solution comes from the business (top) to the physical implementation (bottom).

Solutions Architect must be proficient with the top level: business requirement and strategy, and end to end solution architecture. You should target for level 5 on that. Then going down to the translation of the requirement into the technical solutions. Then going down to how the orchestration can manage multiple domains. Once you are good with all those, you can go down to virtualized infrastructure manager such as OpenStack. If you still have time, build capabilities on VNFs, hypervisor and host operating system. Then all the APIs and interfaces between components. Then programming language, and so on. Try to target the highest level on the top, and moderate level as you go down.

Nothing stopping you to pursue and master physical network knowledge first. But if you are really good in individual technical component, but can’t understand the business driver as well as the overall solution, then you are still at engineer level. A very good engineer perhaps, but not a Solutions Architect.

5. Use Tree Traversal for Development Planning

To give point 4 more concrete example, you may want to use Tree Traversal as I wrote in my previous post. It offers two different approach in visiting each node in a tree data structure:  Depth-first, go down first e.g. first child, then grandchild before second child, and Breadth-first, go across first e.g. first child, then second child before grandchildren.

Your decision to pick which approach, depends on your current situation. Perhaps you are in a work environment where everyone is expected to become a domain expert first, hence you may choose Depth-first. Or perhaps you are being supported to become a Solutions Architect and it is better to start by knowing the whole picture to begin with, then go with Breadth-first approach.

6.  Build Learning Path

How to build learning path for non-technical skills?
Indeed certain skills such as communication, leadership or connecting the dots usually comes with experience. But you can still equip yourself with some books or courses that teach foundation knowledge of those topics. And you don’t have to take MBA to understand the business, but it may help to look at the program curriculum and pursue the knowledge defined in it.

For technical skills, the path can be set by following certification program. Not everyone is a big fan of certification, but the program creator usually spent quite amount of time to define the level of expertise as well as the scope of each level, why not leveraging this instead of reinventing the wheel? And you don’t have to take the exam if you don’t want to, just follow the learning blueprint of the program. You can use Cisco certification for networking, RedHat or Linux Foundation for Linux OS, VMware for virtualization and hypervisor, Mirantis or RedHat for OpenStack and so on.

7. Get Feedback

When you are in process of learning something, it is important to get a feedback to measure your progress. The exam in certification program can be a feedback: if you pass it means you have reached the knowledge level as defined by its creator. Or you may want to get a feedback by getting involved into the discussion. Find out who is the domain expert and try to discuss with him or her to validate your assumption of what you think you already know. Or provide assistance for those who have just started to learn. Seek a feedback by teach the knowledge to others. It was Albert Einstein who said "If you can't explain it simply, you don't understand it well enough."

8. Remove Barriers to Knowledge

I’m just stating the obvious here, but please make sure you have all the access to learn the knowledge. From a simple Internet connection, training material, online library, whitepaper, books, to lab equipment to practice and allocated time and place to study, make sure you don’t have anything to stop you in learning.

9. Make Learning as Habit

In The Power of Habit, Charles Duhigg takes us to the thrilling edge of scientific discoveries that explain why habits exist, and how the right habits were crucial to the success of Olympic swimmer to CEO. There are many examples in the book of how implementing so-called keystone habits can earn billions and mean the difference between failure and success, life and death.

You have to make learning as your habit. And you can do that by starting it slowly, but be consistent. If you plan to spend at minimum an hour a day to learn something new, then do that and stick with it. Later you can increase the amount of time you want to allocate. Later, once it has become your habit, you will find that it is part of your daily routine and something you can’t skip.

10. Take Classroom Training

If you asked me 15 years ago if classroom training is necessary, my answer would be no. Why would you waste your time sitting inside the class, listening to someone who may know only how to teach, instead of doing the reading yourself and trying to apply the knowledge directly? I passed all my CCIE and CCDE exams with a very minimum number of trainings.

But if you ask me again now, my answer is: you may want to consider to go to classroom training. Not because you can’t learn by yourself, but the training can accelerate your learning and shorten the time required to understand the new knowledge. We are all busy so it is good to have someone helping in explaining complex subject to us. Just make sure the instructor is someone who has experience in applying the knowledge in real world, so you will get the knowledge not only the one from the training material.

Another reason is classroom training gives us break from our daily work. You are excused not to reply to your email immediately or to pick up the phone since it is in silence mode: you are inside the classroom listening to the instructor! Your boss or colleague or even customer can be informed in advance and usually they respect your training time and give some space.

11. Exploit Strengths, Manage Weaknesses

By now I believe you have already know what are your strengths and weaknesses. If communication is your strength point, exploit it. Put yourself as the key person to discuss with the customer to get the requirement. So even if technical skills are your weaknesses, you can manage them by delegating the technical tasks to the engineers.

It is important to emphasize that as Solutions Architect your strength point should be the one that is closer to the business on the top. This can cover your weakness at the bottom such as technical implementation, at least for the time being while you are developing the skills. So if you feel that technical detail is your only strength, while you really have to develop your business mindset and communication skills, exploit it by becoming Subject Matter Expert in some technical domains. You are not a Solutions Architect yet, not from the business people perspective, but at least you have a good value to contribute to the team and slowly you can develop the required skills to become one.

12. Believe That It Can Be Done

Last but not least, you have to believe you can achieve your goal. Your mind can play trick with you. Many have not even tried to pursue their goal because they don’t believe it can be done. Skill development plan is important, but what more important is the process in planning it. Start developing the plan and follow it. Achieve small success at a time. Believe it is possible as long as you continue to execute your plan.

Only by believing that he is the one, Neo is able to stop the bullet.

So start building your career strategy now using the points above as guidance. Then implement it with commitment, consistency and determination. Don’t forget to have fun during the process.

Good luck.

“We must learn how to be the CEO of our own careers"
-- Peter Drucker

SDN, NFV and Skill Development for Network Engineers

I came to Cisco Brussels 14 years ago to take my first CCIE lab. I didn’t pass that time. I went to Tokyo a month later and got my number there. Several years later I kept coming back to Brussels to pass my two other CCIEs.

Many people say there is no value anymore in taking CCIE these days. With SDN and NFV, everything will be done "auto-magically". We don’t even need network engineers anymore! Yeah, right. Last week I went back in Brussels to work on Network Control System, multi-vendor network device management tool from Tail-F that was acquired by Cisco June last year. NCS is the cornerstone of Cisco Network Service Orchestration framework for Cisco SDN and NFV solution offering. And I’m here to tell you that the world still need lots of network engineers, and CCIEs, or those who have CCIE-level skill set.

But first, let me talk more about NCS, a service orchestration for real-time service provisioning across multivendor networks.

Network devices were, and are still, configured using CLI. Then SNMP was created to help. Soon, we realized SNMP is great to monitor the network but it fails to become configuration management, as stated by RFC 3535. So more than 70% networks today are still managed by CLI. Arista Networks CEO Jayshree Ullal once said “it’s the way real men build real networks today.” And when we need automation, we create script to send sequential CLI command to devices.

When we use CLI, or Web interface, or SNMP, or send the command over REST API or Netconf to the device directly, this is called Stove Pipe approach. Tail-F NCS puts itself in the middle of the communication between all those protocols with the network devices or Managed Objects (MO). Think its function as mechanism to “normalize” devices configuration into one standard data model then push that configuration into devices using one standard protocol. NCS supports multiprotocol and multivendor by design.

NCS uses YANG, a data modeling language that represents data structures in Extensible Markup Language (XML) tree format. The data modeling language can be used to model both configuration data as well as state data of network devices. YANG can be used to define the format of event notifications emitted by network devices and it allows us to define the signature of remote procedure calls that can be invoked on network devices via the Netconf protocol. Network Configuration Protocol (Netconf) provides mechanisms to install, manipulate, and delete the configuration of network devices. Its operations are realized on top of a simple remote procedure call (RPC) layer. YANG was published by IETF as RFC 6020, and Netconf was published as RFC 4741 and later revised and published as RFC 6241.

In short: The Netconf protocol allows a manager to set configuration, query configuration and state and execute actions on the device (much like SNMP). The YANG models describe everything there is to configure, monitor, admin actions, notifications for each device type and version (much like a MIB).

Many network management tools are misleading since they are actually only device management. What we really need is a service management. When we want to enable service, let’s say MPLS L3VPN, we may have to configure multiple devices. So it’s a network wide transactions based tool that we need. NCS acts as central repository to store network services logically using YANG data model for data structure. The structure represents service instance, and network configuration and state. It then maps between service operation to network configuration changes. NCS has transactional integrity. The service configuration is pushed using Netconf to all relevant network devices as one transaction. So it has to be successful in all, or nothing at all. If one device is failed to accept the configuration change, then all devices must be rolled back to its previous state. And there is no internal order inside a transaction, it is a set of changes, not a sequence. And any parallel transactions do not interfere with each other.

So what’s the relationship between NCS and SDN/NFV?

Cisco has been talking about Evolved Services Platform (ESP), that uses software-defined networking (SDN), Network Functions Virtualization (NFV), Open APIs, and advanced orchestration capabilities to forge a flexible and modular platform. With the Cisco ESP, and the physical and virtual infrastructure service delivery capabilities of the Cisco Evolved Programmable Network (EPN), service providers can quickly deploy new personalized offerings through integrated services modules. Imagine offering prepackaged tiers of enterprise services with default features, security, and service-level agreements (SLAs), that customers can select from an online portal and activate with a click of a mouse.

So the network operator can create a catalog of customized service offers that they make available through the active service catalog. The operator uses the business logic of the network service and resource orchestration modules to start up the virtual machines, activate the Virtualized Network Functions (VNFs), and dynamically create the specific service function chain, which creates appropriate linkages that support the service profile and steer the customer traffic through them. That is the idea behind Virtualized Managed Services solution, Cisco ESP services modules that may be deployed today.

And NCS plays important role as the Orchestration Engine in the solution.

Skill Development for Network Engineers

With the long explanation above, if you are a network engineer, what should you do now? What should you learn? How to develop skills to be able to work with the new networking paradigm like SDN and NFV?

As I wrote in my last blog post, someone needs to translate business requirement into technical requirement. That someone needs to understand business language as well as the technical solutions. Then the same person or someone else needs to create end-to-end architecture just like Virtualized Managed Services. Another person needs to design the service profile and create service catalog. Tool like NCS still requires input from someone to make those services into YANG model. And even if all physical and virtual infrastructure will be provisioned and managed using tool, we still need people who are proficient with network, compute, storage as well as various virtualized network functions. There are still many network protocols today such as OSPF, ISIS, BGP, overlay like MPLS or IPSec tunnel, that will continue to be used in the solution. So you still need to know networking technology in details. We still need many network engineers.

Any SDN/NFV solution and implementation will have so many components connected to each other using different types of API. There are lots of moving parts. Lots of complexity. We still need CCIEs, or rather those who possess CCIE skill set to handle the complexity. Then they have to adapt and to learn something beyond networking.

We need those who want to go up, to get closer to the business, and those who want to have the wide and broad knowledge beyond networking: hypervisor, OS, programming logic, scripting, API and so on. Dave Tucker suggested the transition from NetOps to DevOps, and he would predict that the Network DevOps engineer will need:

• Strong Networking Skills
• Knowledge of Linux System Administration
• Experience with Puppet/Chef/Ansible/CFEngine/SaltStack would be desirable
• Scripting skills in Bash, PHP, Ruby or Python
• Ability to work under Source Control (git)
• Experience in consuming (REST) APIs
• Experience with OpenStack and OpenStack Networking
• Appreciation of Software Defined Networking (SDN)
• Knowledge of Agile Methodologies
• Knowledge of Test-Driven Development
• Ability to write Unit and Integration Tests

With so many things to learn, where and how to start?

Introduce tree-traversal. It is a form of graph traversal and refers to the process of visiting each node in a tree data structure in a systematic way. Basically in the example above there are two ways to traverse a tree: Depth-first, go down first e.g. first child, then grandchild before second child, and Breadth-first, go across first e.g. first child, then second child before grandchildren.

Let’s look at my tree-traversal above. I used Depth-first approach. So I spent my time to learn Networking technology first, in Routing & Switching, then Service Provider and Security. Then I spent time to learn Compute with various Linux OS distributions. And these days I put my focus on SDN/NFV technology from Cloud, Virtualization and network programmability.

So which way is the best for you? It depends.

You may go with Depth-first like me: learn networking until CCIE level, then go deep with Linux, before moving to scripting, API, orchestration and so on. Or you may go with Breadth-first: learning networking to mid level, then switch to Linux, then try scripting a bit, then get some hands on with OpenStack, then coming back to learn networking in more detail, Linux in more detail, and so on. Both ways have pros and cons.

And you may use certification program to help you to learn. For networking, Cisco certification program is still the gold standard. For linux OS you may go with RedHat or Linux Foundation. Even OpenStack has certification program from company like Mirantis or RedHat.

If you do use certification program to learn, please remember one thing: in learning, process matters. At work, obviously outcome or result matters. But when you learn something, don’t focus on the outcome or to get the certification, but focus on the process to gain the knowledge. So you will always be ready to learn any new skills and technology.

And that, I believe, is the only way to become the next generation network engineers.

So stop reading now. And start planning for your skill development.
Good luck.

“Plans are nothing; planning is everything.”
-- Dwight D. Eisenhower

Go Up or Go Wide

You say, the world doesn't need CCIE anymore with the raise of SDN. I say, we still need CCIE, but those who can adapt. You say, my words are just futile last ditch effort to show the importance of CCIE certification. I say, even I still work for Cisco but I don't work for CCIE program, and I get paid not because of my certifications.

If we have data, let's look at data. If all we have are opinions, let's go with mine. 
(Jim Barksdale, former CEO of Netscape )

So let's look at the data to make the discussion more fruitful. If you look at Cisco revenue of each product line for the past 5 years, we see there is decline for NGN Routing and Switching business. And yes, Data Center business is growing in fast rate. And Data Center business includes unified computing, next generation fabric, cloud and most Cisco SDN solutions that are available today.

However, if you do a simple math you can see the combined revenue of Routing & Switching business is still close to half Cisco revenue as of today. We are talking about more than 20 billion USD business. It's declining indeed, but it will take number of years for Data Center business with the same growth rate to reach the same size of revenue.

And below is the revenue comparison between product vs. service.

It shows that the hardware business is declining, but the service business is increasing in size. It can be interpreted freely as there is still demand for those who are able to deliver service. It means as long as you the network engineers don't lock yourself in to particular product, you should be fine. Don't become a product admin but try to really understand the technology. Learn deep about how protocols work not only how to enable the feature.

So I say, if you are still interested to work in computer networking, you can continue pursuing CCIE to gain the skillset. Long gone those days when CCIE gets paid only just because of being certified. You need to be in the right company, you need to be in the right position. You need to be relevant to company's business.

But the requirement for network programmability, virtualization and overlay is real. Those are what SDN has promised even though we are still yet to see the mass implementation of it. Company like Cisco has released its own SDN solution with Application Centric Infrastructure (ACI) and it requires different type of skillset to understand and implement it.

We, as network engineers, must evolve. We, as CCIE, must adapt.

So my message to all CCIE out there: the time has come for you to choose.
Either to go "up" or to go "wide".

We are all here in this field to support company's business. The promise of SDN model is to create abstraction layer to hide the complexity. There is a growing need for customers to put main focus to their core business activities, and let the SDN solution to deal with how to provision and to run the networking infrastructure. There is a dream in the future where customers can just describe their business requirement and what they need from the infrastructure, and let the SDN controller to automatically re-program all the network elements to answer the requirement.

But someone must translate that business requirement into technical requirement. That someone needs to understand business language as well as the technical solutions. That someone has to show the tangible outcome like Capex and Opex reduction that any SDN solution can offer to the business. That someone must be familiar with PERL - performance, efficiency, risk, liquidity of the company, at the same time at least understand the position of PERL language within SDN big picture.

That's go up.

These days I work with several "Cisco" Solutions Architects who posses different type of skillset. Some of them are not even certified in CCNA. They know very little about IGP, BGP and MPLS, but very proficient with hypervisor, OS, scripting, API and SDK. They speak Scrum. They are object oriented type of persons. They always see the overlay and virtual layer while most people still see the physical infrastructure.

Their type of skillset is still incomplete. CCIE type of skillset is far from complete either. Don't expect one type of skillset to dominate the workforce of the future. But imagine the next generation network engineers who possess both types of skillset.

That's go wide.

Possess CCIE skillset only is indeed no longer sufficient. Know only how to pass CCIE exam is even very less useful these days.

Go up or go wide is not similar with red pill - blue pill options. Both options are red pill. Both options are real.

Welcome to the new reality in computer networking.

Disclaimer: Cisco revenue is being used here due to its position as the market leader in computer networking industry. And all the information to make the graphs above is available publicly in Cisco annual financial report. You are more than welcome to analyze other vendor's revenue if you wish.

Some New Old Books

The Lean Startup approach is for companies that want to be more capital efficient and that leverage human creativity more effectively.  It relies on “validated learning,” rapid scientific experimentation, as well as a number of counter-intuitive practices that shorten product development cycles, measure actual progress without resorting to vanity metrics, and learn what customers really want. It enables a company to shift directions with agility, altering plans inch by inch, minute by minute.

The products lean startup builds are really experiments: the learning about how to build a sustainable business is the outcome of those experiments. That information is much more important, because it can influence and reshape the next set of ideas. And it uses the Build-Measure-Learn feedback loop at the core of Lean Startup model.



In short: build Minimum Viable Product (MVP) or Minimum Viable Service (MVS), launch, collect data to learn in order to perfect the ideas. Or if it doesn't work, fail fast, and pivot to another ideas.

Rather than wasting time creating elaborate plans for new product, it's better to launch quickly and find a way to test the idea continuously, to adapt and adjust before it’s too late.

How to decide which ideas to develop first?

In Four Steps to the Epiphany, Steven Blank offers the practical and proven four-step Customer Development process for search and offers insight into what makes some startups successful.

Rather than blindly execute a plan, The Four Steps helps uncover flaws in product and business plans and correct them before they become costly. Rapid iteration, customer feedback, testing your assumptions are all explained in this book.



What will be the next step if we want to use lean principle to launch new product/service?
We still need to create a plan. But don't waste time in creating plan.

Business Model Generation explains the most common Business Model patterns, based on concepts from leading business thinkers, and can help us reinterpret them for our own context. We can learn how to systematically understand, design, and implement a game-changing business model--or analyze and renovate an old one. Along the way, it will make us understand at a much deeper level our customers, distribution channels, partners, revenue streams, costs, and our core value proposition.

Business Model Generation advises us to use "Business Model Canvas" to analyze quickly any product/service we want to build. We can even use a modified model which is the "Lean Canvas" that is closer to the lean startup principle.



Each of us just need to spend few minutes to learn about the Lean Canvas model in order to use it.

Ideas are cheap. Execution is everything.
Think, build, launch. Get feedback, iterate, update. Repeat.