This is why cloud-inspired networks win: virtual switches are all consistent

Since opening up our draft of the 7th edition of Computer Networks: A Systems Approach for comments, we have not yet been inundated with feedback, but we did get a very helpful comment about our perspective on network operations, a subject to which we have devoted an entire chapter.

Both Larry and I have learned over the years that operations are important and yet the topic is frequently neglected when teaching about networks.

Yet the topic is important enough that we even wrote a book on Edge Cloud Operations: As you can tell from the title, that covers more than network operations, but networking is an important part of any cloud. And since we had that material already to hand, it naturally formed the basis of what we wrote in the new textbook edition.

BC can stand for "Before Cloud"

And this is where our helpful commenter pointed out that we’d taken a very specific perspective: our model of operations is heavily influenced by the operational tools and practices of cloud operators, which differ quite substantially from those of more traditional network operators. So that is a gap that we have started to address as we revise the book on its path to publication.

My own networking career can be roughly divided into the “pre-cloud” and “cloud-centric” eras, with the dividing line being my 2011 decision to leave Cisco (the very definition of a traditional networking vendor) for Nicira, an SDN startup. Nicira’s original value proposition was to offer an API to the network that could be plugged into a cloud management platform. More on that in a moment.

I was involved in a few network management projects while I was at Cisco, and I would say that their lack of success told me a lot about why network management has been so hard to tackle in the traditional vendor world.

For a start, there is the breathtaking inconsistency of features across devices. This applies to the products of a single vendor, never mind the variations among vendors. To get a sense of the scope of this problem, look at the documentation for the Network Automation and Programmability Abstraction Layer with Multivendor support (NAPALM) project, a python-based tool that supports programmatic interaction with a variety of networking devices.

NAPALM supports five different vendor operating systems, three of which are from Cisco. And just because two devices have the same operating system doesn’t mean that they will have comparable feature sets, or even that the same feature will behave identically across two devices from the same vendor.

I spent a few years working with some of my peers at Cisco trying to achieve consistency for a subset of features including those related to QoS and MPLS, which was actually a project I’d class as a moderate success. But we had to persuade product teams that having an “innovative” implementation of, say, fair queueing on their product isn’t necessarily a good thing when customers want to make QoS work predictably in a network with dozens or hundreds of devices. Conway’s Law says, roughly, that a company’s products resemble its org chart, and it applies here.

The CLI Never Goes Away

One of my failed projects at Cisco tried to introduce a programmatic API to a range of products so that devices could be configured by software tools rather than by CLI. This was arguably ahead of its time (it was in the late 1990s as I recall) and one of the key reasons it failed was because of the total centrality of CLI management. Every new feature developed on a router had to be configurable via the CLI, or it couldn’t ship. For better or worse, customers expect every feature to be exposed at the CLI. As soon as you asked a product team to support a second configuration interface (an API) you were creating another hurdle for them to cross before the feature could ship. Of course, the product team had no incentive to do this (see Conway’s law) and that is how this project eventually came to a disappointing end.

You could argue that NAPALM has basically stepped up to fill the role that we were trying to impose on product teams, since you can now wrap some Python around a box to expose an API to manage it. And while I don’t have firsthand experience of using NAPALM, that seems to be one of the options that real-world ops teams use today.

Abstractions Matter

A big part of my “conversion experience” that led to my leaving Cisco for Nicira can be traced to the realization that simply putting an API on a box isn’t really a great answer to the problems of network management. As Scott Shenker put it in his influential talk from 2011, what we needed was better abstractions for networks, not just different interfaces to the existing devices. And the way to deliver those abstractions was to place a centralized controller in charge of a distributed set of forwarding devices so that we could think about controlling networks rather than just devices.

What the Nicira team figured out before I joined was that

(a) controlling a network of virtual switches was a simpler task than controlling physical switches, because the software virtual switches could all provide an identical interface to the controller;

(b) solving this problem in the virtual world was a useful contribution, by making datacenter networks easier to manage. Build an overlay network of tunnels between all those virtual switches, and you could neatly sidestep a lot of the problems of managing physical switches: the underlay network remains relatively static while the frequently changing virtual networks can all be managed by the SDN controller. (For a deeper dive into this story see our SDN book.)

One of the challenges we faced at Nicira was finding customers who could successfully adopt a cloud-centric model of network operations. While our product offered a central API to manage virtual networks, that API needed to be plumbed into some sort of cloud management system, and the standard offering at that time was a very immature version of OpenStack.

Running OpenStack was not (and still is not) for the faint of heart. In the early days of trying to drive adoption of our network virtualization platform, our first question to a customer was usually “what cloud management platform do you use” and if we got a blank look then it was time to look elsewhere. (Microsegmentation, not cloud automation, eventually led to widespread adoption of the product.)

So the challenge we face in covering operations for our book is the tension between covering the way we want the world to be versus the way it is. Our current approach, with a focus on OpenConfig and YANG, is much closer to that of hyperscale cloud operators than to the traditional network operators.

This is certainly an appealing way to explain network operations to students so that they can gain an understanding of what problems arise in network operations and see one way to manage the complexity.

To some degree, the hyperscalers can force the world to be the way they want, e.g., by developing their own networking code (see Jupiter, SONiC). This is rarely the case for traditional network operators and enterprises.

So a focus on hyperscale practices is less helpful to the practitioners picking up a book wanting to learn something about how their networks of diverse devices, probably from multiple vendors, can be managed. For this reason we are trying to be honest about the fact that there is no single “correct” way to manage a network, and to cover some of the tooling that applies in the “real world” outside the very prescriptive environments of the hyperscale operators. ®

Larry Peterson and Bruce Davie are the authors behind Computer Networks: A Systems Approach and the related Systems Approach series of books. All their content is open source and available for free on GitHub. You can find them on Mastodon, their newsletter right here, and past columns for The Register here.