Contributing to OpenFlow 1.1 Support in Open vSwitch A report submitted in partial fulfillment of the requirements for the degree of Bachelor of Computing and Mathematical Sciences at The University of Waikato by Joe Stringer c 2012 Joe Stringer Abstract Open vSwitch is a production-quality software switch. It provides high-performance packet-forwarding in virtualised environments. One of the many protocols it supports is OpenFlow, which allows dynamic configuration of network behaviour. Newer versions of OpenFlow have recently been released, adding support for important protocols such as MPLS and IPv6. However, the support for OpenFlow in Open vSwitch is not in sync with the current standards. At the end of 2011, a call for assistance was posted on the Open vSwitch mailing-list, requesting that the OpenFlow community contribute towards support for newer versions of the protocol. This report describes the process of responding to this call, contributing to an open source project, and working towards providing network researchers with a more flexible platform for their experiments. Acknowledgements I would like to take the time to thank the following people for their guidance and support during this project: Richard Nelson, for his supervision of this project; Josh Bailey and the folks at Google, for introducing me to softwaredefined networking; Ben Pfaff, for his continued patience with my enquiries and patch submissions; Justin Pettit, Jesse Gross, Isaku Yamahata, and the rest of the Open vSwitch community, for their help and assistance with reviewing and discussing implementation details; Brendan Jones and Shane Alcock, for their input into this report; And all of the great bunch at WAND, for providing many memorable times. Contents 1 Introduction 1 1.1 1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 1.3 Structure of this Document . . . . . . . . . . . . . . . . . . . 3 2 Background 2.1 Software-Defined Networks . . . . . . . . . . . . . . . . . . . 2.2 OpenFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4 5 2.2.1 Prior OpenFlow Implementations . . . . . . . . . . . . . . 2.3 Open vSwitch . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 2.3.1 Development . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 OpenFlow Testing Framework . . . . . . . . . . . . . . . . . 7 7 3 Development Process 3.1 Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 9 3.1.1 Identifying work . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Learning the Architecture . . . . . . . . . . . . . . . . . . 3.2 Communication . . . . . . . . . . . . . . . . . . . . . . . . . 10 11 12 3.2.1 3.2.2 Distributed development . . . . . . . . . . . . . . . . . . . Ongoing Development . . . . . . . . . . . . . . . . . . . . 12 13 3.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Code Quality Tools . . . . . . . . . . . . . . . . . . . . . . 3.4 Submission . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 14 15 3.4.1 3.4.2 Preparing to submit . . . . . . . . . . . . . . . . . . . . . Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 16 3.4.3 Code Review . . . . . . . . . . . . . . . . . . . . . . . . . 17 4 Existing Architecture 4.1 OpenFlow Architecture . . . . . . . . . . . . . . . . . . . . . 19 19 v Contents 4.1.1 Flow Modification . . . . . . . . . . . . . . . . . . . . . . 19 4.1.2 4.1.3 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . Experimenter Extensions . . . . . . . . . . . . . . . . . . . 20 21 4.1.4 Extensible Matches . . . . . . . . . . . . . . . . . . . . . . 4.2 Open vSwitch Architecture . . . . . . . . . . . . . . . . . . . 4.2.1 OpenFlow Provider . . . . . . . . . . . . . . . . . . . . . . 21 22 23 4.2.2 Testing 4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 25 5 Implementation 5.1 Arbitrary Ethernet Address Masking . . . . . . . . . . . . . 26 26 5.2 Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Matching on Metadata . . . . . . . . . . . . . . . . . . . . 28 29 5.2.2 Writing Metadata . . . . . . . . . . . . . . . . . . . . . . . 5.3 SCTP Support . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 End-to-End Testing . . . . . . . . . . . . . . . . . . . . . . 29 31 33 5.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Code Review . . . . . . . . . . . . . . . . . . . . . . . . . 35 35 5.4.2 Testing features . . . . . . . . . . . . . . . . . . . . . . . . 36 6 Conclusion 6.1 Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 38 39 References 40 List of Abbreviations API Application Programming Interface CPqD Centro de Pesquisa e Desenvolvimento FIB Forwarding Information Base HBO Host Byte Order IETF Internet Engineering Task Force IRC Internet Relay Chat KVM Kernel-based Virtual Machine lksctp Linux Kernel SCTP LSR Label-Switched Router LXC Linux Containers NBO Network Byte Order NXM Nicira Extensible Match OXM OpenFlow Extensible Match RFC Request For Comments SCTP Stream Control Transmission Protocol SDN Software-Defined Networks VM Virtual Machine List of Figures 2.1 OFTest: Black-box testing . . . . . . . . . . . . . . . . . . . . . . 8 4.1 OpenFlow 1.0 Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . 20 4.2 OpenFlow 1.1+ Pipeline . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Open vSwitch Components . . . . . . . . . . . . . . . . . . . . . . 20 22 4.4 OpenFlow Provider . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.1 Ethernet address masking . . . . . . . . . . . . . . . . . . . . . . . 5.2 SCTP test setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 33 List of Tables 5.1 Development breakdown . . . . . . . . . . . . . . . . . . . . . . . . 35 Chapter 1 Introduction Computer networks have become critically important to our lives—the internet pervades the lives of billions worldwide. For network researchers, this provides a mixed blessing: On one hand, innovation and experimentation on networks is more relevant than ever; on the other hand, the scale of installed network hardware sets limitations on the impact that researchers can make in the field. Traditionally, network hardware has been implemented as monolithic boxes with limited configurability. The “Clean Slate” group (Stanford University, 2012) have spent the past five years working towards providing a more open, research-friendly platform for managing network behaviour—OpenFlow (McKeown et al., 2008). This movement has attracted global interest, most notably with Google announcing its use of OpenFlow to increase network utilisation and reduce operating costs (Hölzle, 2012). Open vSwitch (Pfaff et al., 2009) sits at the meeting point of this open networking movement and the recent trend towards increased computer virtualisation. As a software switch that has been designed from the ground up for performance in data center environments, it too has garnered much interest; during the course of this project, the company behind Open vSwitch, Nicira, was acquired by VMware for US$1.26 billion (Herrod, 2012). The availability of these technologies and the open background for them provides a compelling field for research and development. Applications which have previously only been possible through the use of proprietary network hardware are becoming increasingly accessible. One such application is that of a LabelSwitched Router (LSR)—a particular type of network switch that is used in the core of the internet to simplify traffic management. Chapter 1 Introduction 2 1.1 Motivation The motivation for this project arose from the proof-of-concept OpenFlowbased LSR platform developed by Kempf et al. (2011). The use of custom hardware and unstandardised extensions to OpenFlow in that work presents a barrier for research. The custom extensions used by Kempf et al. have since been included in the official OpenFlow 1.1 standard (OpenFlow.org, 2011), and more recently, Google has presented ‘Project W’—a project to make the open source LSR work more accessible. One of the required components for Project W is Open vSwitch. Late in 2011, a call for assistance was posted on the openvswitch-dev mailing-list, seeking support from the community to write code to support OpenFlow 1.1 and above (Pfaff, 2011). Contributing to OpenFlow 1.1 Support in Open vSwitch is a response to this call for assistance, as a step towards providing a more accessible open source LSR. The contributions of this project will also act to benefit the networking research community by improving the foremost implementation of OpenFlow. 1.2 Goals Prior work in this area has required specific hardware and software configurations, which limits the accessibility of the work. To provide lasting value for the research community, this project should focus on contributing work into the upstream codebase rather than providing another proof of concept. Three OpenFlow 1.1 features were selected for this project to focus on: • Arbitrary ethernet address masking • Attaching metadata to a flow • Adding support for Stream Control Transmission Protocol (SCTP) For each of these features, it should be verified that the implementation is accurate to the OpenFlow specification. At the minimum, unit tests should be developed to prove that the resulting switch behaviour is as expected. Any additional testing—for example, using the OpenFlow testing framework (Ericsson Research, 2011a)—provides additional assurance of the protocol compliance. Chapter 1 Introduction 3 1.3 Structure of this Document • Chapter 2 introduces the concept of Software-Defined Networks (SDN), and provides background on OpenFlow and Open vSwitch. • Chapter 3 describes the development cycle of committing to an open source project, with particular reference to Open vSwitch. • Chapter 4 briefly explains aspects of OpenFlow which are relevant to this project and describes the architecture of Open vSwitch. • Chapter 5 discusses the implementation of the OpenFlow 1.1 features specifically targeted in this project and evaluates the success of this project. • Chapter 6 discusses the contributions of this project and possible future additions. Chapter 2 Background This chapter explores the work that this project is based upon; What SDN is and how OpenFlow is related. Previous implementations of OpenFlow are explored, which provides some context for the work on Open vSwitch. Finally, this chapter discusses the plans for developing and testing the functionality proposed in this report. 2.1 Software-Defined Networks The core concept of SDN is quite simple: to be able to configure network behaviour in software. As network hardware stands today, each switch or router (hereafter forwarding element) holds its own configuration and view of global network state. Distributed protocols exist to share this state between forwarding elements, but these operate separately on each forwarding element. It is common for forwarding elements to hold state which is inconsistent with that of its neighbours. Under the SDN paradigm, this state can be held separately from the device. A common case is for a single entity to hold a consistent view of the network and distribute this state to individual forwarding elements. SDN makes a clear abstraction between the layer that makes decisions about how packets should be forwarded (the controller) and the layer that forwards packets (the datapath). The controller keeps track of connectivity information. It builds a map of the network by gathering information about neighbouring devices. Based on this map, it constructs a set of optimal routes for forwarding traffic. The controller then assembles flow entries—a tuple of a packet classifier (match) and how to forward the packet (action). These flow entries are then written to the Chapter 2 Background 5 Forwarding Information Base (FIB)—a lookup table used by the datapath. The datapath is the layer that deals with forwarding individual packets. It holds a lookup table for the flow entries, known as the FIB. Each time a packet enters the datapath, it classifies the packet and performs a lookup in the FIB to determine how to forward the packet—for instance, send the packet out a port. This logic is kept simple to minimise the processing time for each packet. 2.2 OpenFlow OpenFlow is a protocol which follows the SDN paradigm. McKeown et al. describe OpenFlow as an API for the datapath of network hardware, which provides basic building blocks for implementing more complicated functionality in the controller. The canonical use of OpenFlow involves using a simple, commodity switch as the datapath and hosting the controller on a standard PC. Due to the abstraction between forwarding and control, the network performance of the forwarding element is not degraded by hosting the controller remotely (Bianco et al., 2010). Software applications can be created to generate rules to determine network behaviour, while providing little performance overhead. This suggests that OpenFlow could viably be used in production environments as a replacement for traditional network management. 2.2.1 Prior OpenFlow Implementations There have been several OpenFlow software switches developed to date. The initial version from Stanford University was developed to provide a reference that other implementations could follow. This switch was updated for each new version of the specification, to understand the new structures and features; however it did not include implementations of all optional features in later versions of the protocol. It was the reference for implementations up to OpenFlow 1.0, but has not been kept in sync with later releases of the specification. This implementation has been used as a base for several projects: Indigo, a hardware-specific implementation (Big Switch Networks, 2011); an OpenFlow 1.1-only switch (Ericsson Research, 2011b); and OpenFlow 1.2 and 1.3- Chapter 2 Background 6 only versions (CPqD, 2012). Each of these only targets a single version of the OpenFlow specification, and with the exception of Indigo, have no ability to hook in with hardware. Indigo itself is only targeted for physical switch platforms. The call for assistance from the Open vSwitch developers outlined plans to support multiple versions of OpenFlow at run-time, unlike previous implementations (Pfaff, 2011). Erlang Solutions have since released an implementation which also does this, however that project focuses on flexibility over performance. As such, the development of further OpenFlow support in Open vSwitch will provide value to the research community above the existing implementations. 2.3 Open vSwitch Open vSwitch is an open source software switch (datapath) with support for several major protocols including OpenFlow. The codebase is currently over 100,000 lines of C code, contributed by 75 developers. Pfaff et al. (2009) introduced Open vSwitch as an alternative to Linux bridge, for providing highperformance switching between Virtual Machines. The architecture of Open vSwitch is designed such that hardware vendors can write modules that interface with their own physical switches. This allows Open vSwitch to hardware-accelerate switching functionality, and provide the device with OpenFlow support without requiring the vendor to implement OpenFlow. As of this publication, it is known that at least five commercially available switches already support Open vSwitch in this manner. Due to the large community backing of Open vSwitch, it can be reasonably expected that the codebase will continue to be maintained well into the future. The review process for submitting code is fairly rigorous, which makes it more difficult to get code accepted upstream; however, this provides a useful guarantee of code quality. As this project seeks to provide lasting benefit to the research community, the assurance regarding code quality makes Open vSwitch a good fit for implementing an open source LSR. Chapter 2 Background 7 2.3.1 Development Each version of OpenFlow is incompatible with the others; both the controller and the datapath must communicate using the same version. Previous implementations of OpenFlow 1.1 in the datapath have not been written to speak multiple versions of OpenFlow at run-time, which limits researchers to only using controller software that matches the same version of OpenFlow. While this reduces the development time for new protocol support, it also creates division in the base of available OpenFlow software. The approach outlined for Open vSwitch from the openvswitch-dev mailinglist was to instead support multiple versions of OpenFlow at run-time. The plan for this is to create generic structures that abstract the protocol differences from the core code, then implement the specific protocol handlers separately. For each new version of OpenFlow, a new parser will need to be added and taught how to convert to and from the internal format. The benefit from this approach is that Open vSwitch can be used in a wider range of environments than datapaths that only support a single version. In addition to this, the plan specifies that newer features should be backported to the older protocol, by adding them as experimenter extensions. This approach will allow Open vSwitch to interoperate with a wide range of software and hardware configurations. Section 4.1.3 describes the usage of this OpenFlow feature in this project. 2.4 OpenFlow Testing Framework Just as there multiple versions of the reference OpenFlow switch have been developed, there have also been multiple versions of a matching test framework, known as OFTest (Big Switch Networks, 2012). This framework is written in Python, and conducts black-box testing of a switch: It surrounds a datapath, controlling all of the inputs and outputs. Figure 2.1 shows this relationship between OFTest and the datapath that it tests. OFTest acts as the controller for the datapath by sending OpenFlow messages to it, and also connects to the ports on that datapath to send packets through it. The behaviour of the switch can then be evaluated against the OpenFlow standard. Tests are conducted independently by conducting the following: 8 Chapter 2 Background OFTest Server OpenFlow connection Datapath under test Test traffic Figure 2.1: OFTest: Black-box testing • Reset the rules on the test switch • Install new flow entries into the datapath • Send packets through the datapath ports • Observe the behaviour This provides an external test for protocol compliance—a level of guarantee about interoperability with other OpenFlow implementations. It is aimed at providing OpenFlow developers with a simple ‘plug-and-play’ environment that is easy to set up and use. Section 5.4.2 will discuss the use of OFTest to evaluate this project, and the issues faced in this process. Chapter 3 Development Process Contributing code to an open-source codebase requires additional skills on top of the basic skillset of a programmer. While the ability to reason about programs, select appropriate data structures and write quality code are all very useful skills, these need to be supplemented with a social understanding of the project’s ecosystem. This chapter describes the considerations that a potential contributor (hereafter the contributor) needs to make when undertaking work on open source software. This will include general commentary on each particular aspect of the development process, followed by examples of how the aspect is managed in Open vSwitch. This will provide insight into the processes involved with a large community consisting of dozens of contributors, and should be widely applicable in any software development community. The general development process can be broken down into a cycle involving the following activities: • Learning about how to contribute • Communicating plans for contributing • Implementing the functionality • Submitting the work upstream 3.1 Learning There are a few steps that the contributor must take to learn how to contribute to an open source project. Learning about what modifications are being sought Chapter 3 Development Process 10 after, and the desired approach for this implementation is one step. Once it has been established what contributions that the contributor will make, the architecture must be examined and learnt. This provides the basis for the work to be carried out. 3.1.1 Identifying work This step can be split into two parts. The first of these is negotiating with other developers about what tasks to work on, preventing duplicate efforts; the second is to understand what is expected of the final implementation of the feature. In the case of implementing a standardised protocol, this involves studying the specification to ensure that the work accurately reflects the intended behaviour. Deciding which feature to work on is the first step in making a contribution. It is recommended that new contributors select something reasonably small at first, so that progress can be made without becoming overwhelmed with the scale of the codebase. With Open vSwitch, the call for assistance provides a useful basis for feature selection. As one becomes more familiar with a codebase, it becomes easier to consider and understand the effects of implementation decisions on other areas of the code. In any project that has a large number of active developers, there are likely to be several features being worked on at a given time. To avoid duplicate efforts, it is important to make contact with the lead developers and determine whether there is already a developer allocated to the feature. In the case of Open vSwitch, the developer mailing list is the central point for such communication (Open vSwitch, 2012b). The contributor expresses interest in developing a particular feature, and other developers provide feedback about whether the feature is already under development. This may also include some indication as to how they would prefer the work to be carried out. When a decision has been made with regard to the feature to focus on, there is a process of learning the requisite implementation details of the feature. This may involve searching for literature on the subject—reading papers, specifications and IETF Request For Comments (RFC)s. In the case of implementing OpenFlow features, the primary source of information is the specification distributed by the Open Networking Foundation. Additionally, many of the features make reference to protocols which are defined elsewhere, so it is Chapter 3 Development Process 11 important to have an understanding of the operation of these protocols. 3.1.2 Learning the Architecture Having investigated what modifications should be made to the codebase, the contributor must learn about how to make these changes. Several factors affect the significance of this step. For some idea of the scope, one could consider how many lines of code there are, or the number of files in the codebase. For a more accurate gauge of the effort required for this task, it is worth looking at the quality of documentation—to what degree it exists and how accurate it is. There is no single accepted format for the documentation of a project. Most codebases have some form of web presence—particularly open-source ones— but even so, the documentation may not be found on the website. In Open vSwitch, the developers have opted to distribute documentation with the codebase, and make these accessible from the main website (Open vSwitch, 2012a). The descriptions of the abstraction layers are found in the PORTING file, which is targeted towards developers interested in porting the codebase to additional platforms. This is augmented by the descriptions of design decisions made when developing Open vSwitch, which is detailed in the DESIGN file. Both of these files are found in the root directory of the codebase. Even when there are design documents distributed with the project, they may not provide the level of detail required to implement a feature. The PORTING guide gives a useful overview of the Open vSwitch architecture, but it does not mention the specifics of how the existing components parse OpenFlow messages or how these messages are implemented. One example is the meta-flow structure which assists with parsing OpenFlow matches at run-time. Such structures can be learnt by searching for components which interact with them and investigating the interactions, and through communication with the upstream developers. Lastly, the nature of working on a fairly new codebase is that the internal data structures have not been fully stabilised yet. Prior to this project, much of the internal representation of data was based directly on the OpenFlow 1.0 structures. Over the course of this project, the internal structures were adapted to allow for differing representations and additional features. Most notably, the internal representation of OpenFlow actions was introduced as a superset Chapter 3 Development Process 12 of the OpenFlow 1.1 actions and instructions so that Open vSwitch could dynamically support multiple versions of OpenFlow at runtime. Such evolution in the codebase will affect the implementation of both new and existing features. As such, the contributor needs to follow developments throughout the entire project codebase to determine the effect that they may have on the proposed code modifications. 3.2 Communication Learning to communicate appropriately is one of the most important aspects of contributing to an open source codebase. Healthy communication can lead to a streamlined development pipeline and less wasted effort. These provide positive reinforcement and improve the pace of community contributions. This is particularly important with a large project such as Open vSwitch, which receives upwards of twenty posts on the developer mailing-list per day. Any of these could affect the implementation of a feature that the contributor is working on. 3.2.1 Distributed development With any existing development community, new contributors need a support structure to help them to develop effectively on the codebase. This structure provides information about the architecture, insight into design decisions, and commentary on developments. When working with a distributed development team, the ability to use other contributors as a resource is hindered by the geographic spread of the community. Typically, open-source codebases with a significant number of contributors have a large variation in development locations. Bird and Nagappan (2012) investigated the geographic distribution of two major open-source communities, Firefox and Eclipse. They found that around a third of major contributions were made by developers on different continents for Firefox versions 1.5 and 2.0. Eclipse was far less distributed, but still involved two continents for a majority of the work. In Open vSwitch, the majority of commits are contributed by Nicira in California. As the software continues to be developed, major contributions have begun coming in from other locations. For the work described later in this document, Chapter 3 Development Process 13 the majority was developed in NZST (UTC+12), while corresponding with California in PDT (UTC-7) and Japan JST (UTC+9). These timezone differences have a considerable effect on how developers can schedule their workflow. While the direct implementation work can be carried out at any time, discussion with developers on the implementation can only occur during set times of the day. The common case for this project involved using the IRC channel for urgent communication in the morning NZST, or using the mailing list for more detailed discussion on implementation or code quality. To set reasonable expectations for what defines a timely response, it is recommended to be aware of what times of the day are business hours for other developers. Upstream developers may take a day or more to respond to messages that are posted to the mailing-list. As such, it is also recommended for the contributor to have additional tasks to work on while waiting for responses. 3.2.2 Ongoing Development Every development community will have a commonly agreed means for communication. Common forms are the mailing list, bug tracking systems and IRC channels. For Open vSwitch, the primary communication platform is the mailing lists. There is a list specifically for development discussion, which is separate from the general user discussion. Bug reports are also posted on a Debian Linux bug tracker, but there is no public bug tracker specifically set up for Open vSwitch. To ensure that they reach the attention of the appropriate developer, bug reports are expected to be reported to either the developer list or the Debian tracker. It is important as a contributor to actively watch the development activity to promote co-operation between developers on related features. Every feature described in this report was affected by other activity on the list. Examples include new abstractions introduced in the core codebase, commentary on related features, and general development guidelines. In particular, the write-metadata feature—described in Section 5.2.2—was based upon another patchset from the community. Active participation on the mailing-list also allows for the contributor to provide assistance to other developers. Some of the work submitted from this project influenced other development. For example, the initial write-metadata Chapter 3 Development Process 14 patch provided a model for others to implement similar features (Pfaff, 2012a). The discussions following the post provided additional insight into the preferred approach for implementation. This is useful to the contributor to improve the understanding of the codebase. 3.3 Implementation There are several ways to verify the code that the contributor has created is of acceptable quality to be included upstream. Perhaps the most indicative is to write a comprehensive set of tests to verify that the behaviour is as expected. Often, software projects will have additional criteria to keep the code maintainable and find unexpected bugs. The Open vSwitch codebase includes a guide for submitting patches which clearly outlines the quality criteria for code submissions. The minimum expectations are that the code: • builds correctly, • does not break any existing tests, • changes a single feature, • and updates the user documentation. This section will cover the expectations for proving the functionality of the implementation, which will be followed by a discussion of its presentation in Section 3.4. 3.3.1 Code Quality Tools For code to be accepted upstream, it must first meet particular quality criteria. As code submissions can be sent by a programmer of any skill level, it is useful to have tools to verify the quality of the code. This assists the contributor in improving code programming practice, and assists upstream developers by providing independent verification of code correctness. Two tools are used by Open vSwitch developers for this purpose: Sparse and GNU AutoTest. Sparse is an open-source static code analyser for C. It is a frontend for popular compilers that provides additional information about the use of memory address space and other resources. In Open vSwitch, this program is used to detect some potential bugs such as mixed byte ordering. When dealing with Chapter 3 Development Process 15 network packets, the endianness of data on the wire is often different from that of the host—referred to as Network Byte Order (NBO) and Host Byte Order (HBO). Open vSwitch differentiates these types by defining its own types for NBO values (ovs_beXX), and using standard UNIX types for HBO (uint_XX); however, the base data type of each of these is the integer. Typical compilers will not detect errors where a developer is directly copying values between NBO and HBO variables without swapping the endianness. Sparse is built to highlight these difficult-to-detect mistakes. AutoTest is a framework for generating platform-specific test scripts. Developers create AutoTest scripts which define general information about a test, how to run the test, and what the expected output is. The Open vSwitch codebase uses AutoTest as the basis for most of its unit tests. When adding functionality to the code, developers are expected to create additional tests to prove the behaviour of the additions is correct. This also allows developers to verify that there have been no regressions when further changes are made to the codebase. When a feature has been written and appropriate test cases have been created, the contributor must then prepare the work for review. Firstly, it must be checked that the patch compiles successfully against a fresh copy of the codebase. This can reveal build errors where the contributor has neglected to include all of the relevant modifications in the patch. Following this, the code should be regression-tested. Running make distcheck in Open vSwitch will perform these two tasks in a single step. 3.4 Submission There are several considerations for the submission of a patch beyond the code implementation. Broadly, this section will describe them in terms of modifications required by the contributor, licensing considerations and feedback from upstream maintainers. 3.4.1 Preparing to submit The Open vSwitch codebase includes a style guide which defines the format which is expected for code before it is to be accepted upstream. In practice, this is unlikely to be an issue if the developer follows the existing style from Chapter 3 Development Process 16 code surrounding his modifications. Given a sensible patch, the upstream developers are unlikely to decline to review based on style but it is likely to draw attention away from the functionality. When the patch modifies or introduces new user-facing features, it is also expected that the user documentation is updated. For each of the userspace utilities provided by Open vSwitch, there are Unix-style manual pages describing the features of the utility and the syntax for using it. A submission that does not include or update the manual pages will not be accepted upstream. It is important that the behaviour described by the manual is accurate to the behaviour of the application. Patches must also be signed off by the contributor, to indicate that the code can be submitted to the project without breaching the licensing terms. It is a public record of the contribution. The usual case is where the contributor has written the code, although it may include cases where code from another source has been included under a compatible license—as described in Section 3.4.2. Lastly, it is expected that any new patches are based on the latest development code branch. It is good practice to fetch the latest version of the development codebase before applying the candidate patch and performing regression testing. This is because the latest code may modify the components that the candidate patch depends on. This is particularly important for projects which have a high volume of code contributions. 3.4.2 Licensing Open-source applications are defined as such based on the license under which they are distributed. On occasion, it may be sensible to include another project’s code in a patch so as to reduce duplicate efforts. This allows the contributor to take advantage of a stable existing codebase rather than reimplementing a new—and potentially buggy—solution. In this case however, it is essential to be aware of the license restrictions on the codebases involved. In this project, the primary example of needing to include external code is the CRC32c checksum implementation for userspace support of SCTP. To reimplement a checksum algorithm is a risk-prone undertaking and would require a large amount of effort to write correctly. As such, it is sensible to look for existing implementations that could be included in the Open vSwitch source. Chapter 3 Development Process 17 Several open-source kernels include copies of the CRC32c algorithm for use by their own SCTP implementations. Each of these has its own license, some of which are compatible with the Open vSwitch license and some which are not. Ultimately, the FreeBSD implementation was chosen, due to its licensing compatibility with the Apache version 2 license that most of Open vSwitch uses. 3.4.3 Code Review The review process is the last step in the development cycle. For Open vSwitch, this step involves the contributor posting the candidate patches to the openvswitch-dev mailing-list, and receiving feedback from other developers. This step provides an additional assurance of code quality for the codebase. There are some measures of code quality that cannot be measured using static code analysis or written tests. One such measure is of the effect on the architecture of the system. The core developers of a project are able to provide insight into how a particular work interacts with the larger codebase. As maintainers of the code, it is in their interests to ensure that code follows the intended design of the system. Conversely, if they allow code that breaks down the design or behaves unexpectedly, then this will increase the work required in the future to keep the system bug-free. For the contributor, the most beneficial aspect of this step is an opportunity to discuss implementation with direct reference to code. Although questions can be asked on IRC or on mailing-lists without a patch submission, the lack of context can introduce ambiguities in communication. When the contributor submits a patch, this is a chance for the upstream developers to educate the contributor on how the work should be carried out. The review process commonly repeats multiple times until the upstream developers are content with the proposed changes. If the contributor is persistent and willing to respond to feedback, then each round of review will bring the code closer to acceptance upstream. Kernel code Open vSwitch also includes a kernel module for Linux in its distribution, which has been included in the main distribution since version 3.3 (Calleja, 2012). The review process for this code is more rigorous—the patch must also satisfy Chapter 3 Development Process 18 a further 24 patch submission criteria (Riel, 2006), and will be seen by multiple other developers. As such, features that require work in this area will undergo additional scrutiny from the Open vSwitch developers, to reduce the effort required to later get the code included in Linux. The SCTP feature described in Section 5.3 requires work in this area. Chapter 4 Existing Architecture Chapter 2 discussed the concepts underlying this project: SDN, OpenFlow, and the approach to extending Open vSwitch. The overview of these concepts is essential for understanding how this project fits into the larger work, but does not provide enough detail to understand the proposed changes. This chapter describes some of the implementation details that this project relies upon; the inner workings of OpenFlow switches in general, and the way that Open vSwitch reflects these. This provides a basis for the modifications detailed in Chapter 5. 4.1 OpenFlow Architecture Prior descriptions of forwarding elements in this document refer to the forwarding of individual packets. However, to place one flow entry into the FIB for each packet that passes through the datapath would be inefficient. As such, packets are usually classified into flows based on attributes such as the source, destination and protocol being used. All packets with the same values in these limited fields is handled by a single rule. 4.1.1 Flow Modification In the OpenFlow 1.0 specification, the datapath’s FIB is presented as a single logical table. The table has a list of flow entries describing the flow to match on, and a set of actions to perform on packets that match. At the bare minimum, an OpenFlow switch must support actions to either forward or drop the packet; there are also provisions for modifying the packet and queueing. When a packet is received, the headers are parsed and used to find 20 actions matches Chapter 4 Existing Architecture Flow Table Figure 4.1: OpenFlow 1.0 Pipeline a matching flow entry. The actions for that flow entry are then immediately applied in order. The default action if there is no flow entry is to forward the packet to the controller so that it can create a flow entry to match the packet. Figure 4.1 contains a simplified representation of the OpenFlow 1.0 pipeline. The arrow represents a packet being received by the datapath. When the packet is received, fields from the packet are looked up in the table to find the associated action. The action is then taken—for instance, forwarding out a particular port. 4.1.2 Instructions In OpenFlow 1.1, the datapath is expanded to support multiple tables. The pipeline processing begins at the first table, and may finish processing after a single table (as with OpenFlow 1.0), or may be configured to continue Table 0 ... instructions actions + metadata matches instructions matches processing on another table. Table N Figure 4.2: OpenFlow 1.1+ Pipeline Chapter 4 Existing Architecture 21 Instructions in OpenFlow 1.1 allow configuration of this processing behaviour. They take the place of actions in a flow entry, providing a superset of the functionality. Instructions may have actions attached, in which case they specify how those actions are to be performed; for instance, the Apply-Actions instruction performs the attached actions immediately—in the same manner as OpenFlow 1.0. Other instructions provide for attaching actions or data to a packet, and moving the processing immediately to any later table. Figure 4.2 displays the pipeline for OpenFlow 1.1 and above. As with the OpenFlow 1.0 pipeline, the fields from a packet are looked up in a table, starting with the first. The table will provide instructions for further processing. This may forward to a later table, or it may immediately apply a set of actions. If the packet is forwarded to another table, then actions and data can also be attached to the packet as it goes through the pipeline. 4.1.3 Experimenter Extensions As a protocol developed in the research community, OpenFlow has always had an inherent focus towards assisting experimentation. Early development of the specification was done in the open. By version 0.89, there were provisions specifically for allowing vendor extensions, later known as Experimenter Extensions. The experimenter extensions support consists of standardised structures for adding new matches, actions and instructions (OpenFlow.org, 2011). This allows researchers to develop functionality to be included in subsequent versions of the specification. Vendors and researchers may use this feature to build upon the existing feature-set from the standard protocol. Many of the features from Nicira’s extensions to OpenFlow 1.0 have become standardised in later versions of the specifications. One such example is the extensible matches, which are used by all of the features described in Chapter 5. 4.1.4 Extensible Matches Extensible matches provide researchers with additional flexibility for modifying flow entries. In OpenFlow 1.0 and 1.1, flow modification messages contain fields for every type of match that the protocol supports. When constructing these messages, the match entries for all of these features must be sent, even if the flow entry only matches on a single field. Extensible matches allow 22 Chapter 4 Existing Architecture these messages to only include relevant matches. This feature was introduced in Open vSwitch as Nicira Extensible Match (NXM), and was standardised in OpenFlow 1.2 as OpenFlow Extensible Match (OXM) (Open Networking Foundation, 2012). To support new features on all versions of OpenFlow, these structures will need to be updated. 4.2 Open vSwitch Architecture The internal structure of Open vSwitch consists of the following components: • ovs-vswitchd: the daemon process that stores state, • netdev-provider: provides network device information, • ofproto: brokers OpenFlow connections, • ofproto-provider: implements OpenFlow functionality. Figure 4.3 shows the relationship between these components. ovs-vswitchd is the main userspace program, which stores and retrieves state from a database. The netdev-provider handles interaction with network devices, which is done by interfacing with the running kernel. ofproto communicates with the OpenFlow controller, and passes messages down to the ofproto-provider. Finally, ofproto-provider is the component which understands and implements OpenFlow—parsing the messages and installing rules to forward flows. ovs-vswitchd ofproto netdev-provider ofproto-provider kernel Figure 4.3: Open vSwitch Components 23 Chapter 4 Existing Architecture 4.2.1 OpenFlow Provider This project focuses on implementing features in the OpenFlow provider. Figure 4.4 shows the primary functions of this component. The parser takes OpenFlow messages from the higher level, and converts them to the internal flow entry format. This flow entry is then written to the datapath, which performs the forwarding of packets. The OpenFlow provider distributed with Open vSwitch contains two datapaths: one in userspace, and one in the Linux kernel. These datapaths make use of a shared OpenFlow implementation for features which they do not directly support. The default behaviour is that Open vSwitch will attempt to use the kernel DataPlane for forwarding, as this is faster—packets do not need to cross the userspace/kernelspace divide. In cases where the kernel implementation does not provide particular OpenFlow functionality, it will revert to using the userspace library. 4.2.2 Testing Open vSwitch includes an extensive test-suite for over 1000 individual features. The testing of OpenFlow features can be broken up into the two main functions from the OpenFlow provider—parser and DataPlane. The parsing tests just check that the OpenFlow messages can be understood and converted into the OpenFlow 1.0 OpenFlow 1.1 Nicira Extension parser Match Flow Table Action ... Flow Table datapath Figure 4.4: OpenFlow Provider Chapter 4 Existing Architecture 24 internal format; the DataPlane tests ensure that the functionality of these messages matches the specification. These tests are written with reference to the internal workings of Open vSwitch, known as white-box testing. Testing the Parser Open vSwitch includes a commandline tool called ovs-ofctl that can parse OpenFlow messages and print text reflecting the purpose of the message. The contributor specifies the hex bytes of the message as input, then ovs-ofctl parses the message as though it had been received from the controller. For a test case, a test’s expected output will initially be set to expect an error message for the feature being tested. This shows that Open vSwitch will notify the controller when a message is sent that uses an unsupported feature. When support for the feature is written, the contributor adds the new feature to the print component. Subsequently, the contributor updates the expected output to the new feature to show that the parser recognises the new feature. Testing Functionality The functionality of a feature is tested in a different manner: by setting up a copy of Open vSwitch and sending packets through it to determine whether the behaviour is as expected. Unlike the OpenFlow Testing Framework described in Section 2.4, these tests build upon knowledge of the internal structures to test that the behaviour is correct. The test script starts Open vSwitch and carries out the following: 1. Send a flow modification message including the new match and an action 2. Send a packet which satisfies the match criteria 3. Verify that the specified action is carried out Between the parsing tests and the functionality tests, the Open vSwitch test suite covers the interpretation of OpenFlow messages and implementation of their behaviour. The use of this test suite to test the features implemented in this project is discussed further in Chapter 5. Chapter 4 Existing Architecture 25 4.3 Summary This chapter has described the internal structure of how OpenFlow switches— and in particular, Open vSwitch—provides packet-forwarding functionality. The next chapter will build upon this knowledge to describe the work carried out in contribution to OpenFlow 1.1 support in Open vSwitch. Chapter 5 Implementation This chapter explores the work carried out for this project: support for arbitrary ethernet address masking, attaching metadata to flows, and implementing support for the Stream Control Transmission Protocol. Each feature is explained separately in the following manner: the feature is described in context of network research—what the feature is and how it can be used. The approach for implementation of the feature is then described with reference to Open vSwitch architecture. Each section will conclude with a description of how the feature was evaluated. Finally, this chapter will evaluate the success of this work based on the original goals. 5.1 Arbitrary Ethernet Address Masking In IP-based networks, it is common to use an address comparison method known as address masking. This allows forwarding elements to use part of an address to direct network traffic. The OpenFlow 1.1 specification adds this feature for use with ethernet addresses (OpenFlow.org, 2011). When the controller adds a flow entry to the Forwarding Information Base (FIB), it specifies an address and a bitmask to match on. Then, when a packet enters the pipeline, the forwarding element checks that the specified mask bits are the same between the flow entry address and the packet’s address. Figure 5.1 shows an example of a positive ethernet address match. 0xFF specifies that the corresponding bits should be identical in the match address and the packet address. 0x00 specifies a wildcard; the corresponding bits need not match. Traditionally, this functionality has not been provided by ethernet switches. In the interests of providing additional flexibility to researchers, the OpenFlow 1.1 27 Chapter 5 Implementation match address: with mask: 01:23:45:67:89:AB packet address: 01:23:45:AB:89:67 FF:FF:FF:00:00:00 } match entry Figure 5.1: Ethernet address masking standard makes these addresses arbitrarily maskable. Particular sections of the ethernet address have significance, such as identifying the sending device or determining whether the address is globally unique (IEEE Computer Society, 2002). This allows researchers to investigate the behaviour of particular devices in their networks. Prior to this project, the Open vSwitch codebase provided limited functionality for masking ethernet addresses. As an OpenFlow 1.0 compliant switch, it allowed exact matching of ethernet source and destinations. Additionally, Nicira have added extensions to match the ethernet multicast bit—a section of the address that specifies whether the destination is a group of devices or just a single device. The existing support for masking was implemented with a set of statically defined masks that indicate the following mask types: • Mask all bits—Match a specific ethernet address • Mask all bits except multicast—Match an ethernet address, but allow it to be directed at one or more destinations • Mask the multicast bit—Match all traffic directed at multiple destinations • Mask no bits—Do not match the address The NXM match message for ethernet source and destination addresses had a field to specify which of these masks should be applied. When the message was parsed, the enumerated mask value would be placed into the flow structure, which would be used by the classifier to dynamically apply the appropriate mask. Rather than applying pre-determined flow masks, this feature allows for specific masks to be attached to the flow entry. The work carried out in the userspace datapath was primarily in the classifier and flow components. A Chapter 5 Implementation 28 new field was added to the flow structure to store the ethernet mask and the classifier was updated to use these new fields. In the OpenFlow parser, the NXM message handler was modified to use the new mask field rather than the previous masks. Additionally, many of these functions implemented their own bitwise operations to modify and compare multi-byte ethernet values. To reduce duplicate code and increase the readability of functions that use this code, it was deemed worthwhile to refactor it into separate functions. The testing for this feature followed the description in Section 4.2.2. The ovs-ofctl tool was used to parse a flow modification message containing an ethernet address and a previously unsupported mask. For it to print out the address and mask correctly shows that the parser has correctly interpreted the message and placed it into the internal structures. The classifier tests were also updated to use the new mask structures, and confirm that rules which match on arbitrary ethernet masks would take action when the mask matched a packet being received. The implementation of this feature was considered as an opportunity to explore the codebase and become familiar with the processes as described in Chapter 3. As the first feature written as part of this project, the implementation took longer than originally scheduled. These delays can be categorised into two areas: Inexperience with the large codebase, and lack of familiarity with the code submission process. The knowledge gained in these areas improved the development speed for the subsequent features. 5.2 Metadata The metadata feature in OpenFlow 1.1 allows additional information to be attached to a packet as it passes through the forwarding pipeline. One table could perform classification of a packet based on particular protocol fields and pass this information to later tables. This data can then be used to perform actions later in the pipeline. A similar feature was previously introduced to BGP (Traina et al., 1996). Donnet and Bonaventure found that on average this feature was being used by half of the routes held by the sampled set of routers, and this number was increasing over the period they were monitored. This gives some indication of the value that a similar feature is providing to Chapter 5 Implementation 29 network administrators. 5.2.1 Matching on Metadata The datapath implementation for matching on metadata is similar to that of ethernet matching—Add a field to the flow structure and update the functions which pack and unpack this structure. After the lessons learnt from the ethernet matching work, these areas were well understood. This increased the speed of implementation initially; however, the work required in the parser differed significantly. This was because the ethernet mask implementation had some existing code to extend, whereas the metadata feature did not build upon an existing feature. The two standardised representations for metadata are the OpenFlow 1.1 match type and the OXM type used in 1.2 and above. The parser for each of these representations was updated to handle the new field. In the case of the extensible match parser, the existing implementation only handled the NXM format and any structures in the OXM format that mirrored the NXM counterpart. Prior to this project, the parser would only handle these messages if there was a representation in both formats. Duplicating the same structures for new features in both NXM and OXM provides no benefit, and is only used for older features for backwards compatibility. As such, it was deemed worthwhile to extend the extensible match support so that new OXM structures could be used with the Nicira extension format. To test the new field, some tests were written to send metadata match messages to the datapath and check that the parser recognises the messages correctly. This was done for each of the possible representations, including the use of the new OXM metadata structure within a Nicira match message. Unfortunately, while the current version of Open vSwitch implements multiple tables internally, it does not support the OpenFlow 1.1 instruction to direct the pipeline to a later table. As such, the matching functionality of transferring metadata to another table was not unit-tested. 5.2.2 Writing Metadata The initial approach to writing metadata was performed using a Nicira action extension, NXAST_REG_LOAD. This extension allows modification of any field in the flow structure. Each field from the flow structure is represented by Chapter 5 Implementation 30 a number, designated by the NXM and OXM implementation. This is used to identify which flow field that NXAST_REG_LOAD will change. This action made it possible to independently test metadata matching before developing the write-metadata instruction. Later, the behaviour of the write-metadata instruction could be compared against this to determine whether the behaviour was correct. During the implementation of this feature, a new abstraction for actions was introduced internally. The new format abstracts away the difference between actions and instructions from the core code, leaving it up to the parser to translate these structures into specific OpenFlow versions. This keeps the datapath code simple, as it does not need to handle the specifics of which OpenFlow version is being used. Instead, the conversion between the internal format and particular OpenFlow versions is handled in the parser, as described in Section 4.2. This approach to implementation has curious effects when combined with the outlined plan to support new features on OpenFlow 1.0 with Nicira extensions. The OpenFlow 1.1 specification defines the order in which instructions should be applied; however, actions do not have the same guarantee. This is further complicated when one considers that the same Nicira extension message could be attached to an OpenFlow 1.1 message in addition to the official instruction. After a discussion with the Open vSwitch developers, it was determined that the best approach is to enforce specific rules regarding the action’s position in a message (Pfaff, 2012b). OpenFlow 1.1 instructions that are implemented as Nicira extension actions must occur at the end of a flow modification message, and must appear in the correct order—as specified for OpenFlow 1.1 instructions in the specification. Furthermore, an OpenFlow 1.1 or higher message cannot contain the metadata action structure. Any flow modification message that does not comply with these restrictions should be responded to with an error message. Testing Metadata Writing Testing the write-metadata instruction is performed using the OpenFlow packet-in message. This message is usually used when a packet enters the datapath and there is no flow entry that matches it. In this case, the OpenFlow switch encapsulates the incoming packet and sends it to the controller with Chapter 5 Implementation 31 any additional contextual information that it can gather. The controller may then investigate the packet and create a rule to match it. In Nicira extensions (and later, OpenFlow 1.2), this message also passes back any metadata that is attached to the packet. A flow entry can also explicitly specify that matching packets should be sent to the controller. This particular behaviour is used to test the writing of metadata. The test procedure is carried out as follows: Firstly, a flow entry is installed on the datapath to match all traffic and apply the following actions: • Attach some specific metadata value to the flow • Send the packet back to the controller Secondly, a packet that matches the flow entry is sent to the datapath. The datapath matches the packet, and performs the actions specified—attaching metadata and sending a packet-in message to the controller. The message can then be examined to determine whether the metadata was written as expected. There was a particular counter-intuitive test case with the expected functionality. There are no instructions in OpenFlow 1.0, so the write-metadata instruction is implemented as a vendor extension to the actions. When ovs-ofctl is used to test the conversion of this message to a OpenFlow 1.0 action structure, the tool outputs a Nicira extension structure. However, when the same is done for converting to a OpenFlow 1.1 structure, the metadata is not displayed. This could be mistaken as an example of the OpenFlow 1.1 parser failing to parse instruction messages correctly. The actual case is that in OpenFlow 1.1, this message will only be represented as an instruction. This test case only uses ovs-ofctl to convert into action structures, while the actual behaviour in practice is that an OpenFlow 1.1 message will be parsed for both instructions and actions. Hence, the expected output from the test is actually to drop the action. 5.3 SCTP Support SCTP is a transport protocol similar to TCP or UDP (Stewart, 2007). SCTP is used by the telecommunications industry to interconnect various systems in their networks, in particular newer billing and authorisation systems (Calhoun et al., 2003). As a relatively recent protocol (first introduced 2000), it is of Chapter 5 Implementation 32 interest to researchers to experiment with its behaviour and compare it to traditional transport protocols. The implementation of this feature was split into four areas: introducing a CRC32c checksum implementation for use in the userspace datapath, parsing OpenFlow matches for SCTP, and support for matching SCTP in each of the userspace and kernelspace datapaths. Section 3.4.2 described the considerations for including a CRC32c implementation, and the previous two sections outlined the modifications required to support a new OpenFlow match type. Therefore, this section will focus on the implementation of SCTP in the datapaths. The SCTP support in OpenFlow 1.1 refers to the ability to perform matching and actions using SCTP source and destination port numbers. These port numbers allow hosts to distinguish between multiple connections with the same destination. This functionality mirrors that of several other protocols, including TCP and UDP. Furthermore, the location within a packet for these port fields is also identical. As such, some of the functionality for this feature was implemented in the same format as the code for these other protocols. This was the case for the matching support; the logic consists of looking at a particular byte offset into a packet to find the port, and using the value to compare with the relevant flow entry. The feature support that deviates the most from these other protocols provides support for the set-field action. This action allows particular fields of a packet to be altered by the datapath as it passes through the pipeline. When altering part of a packet in this manner, the checksum for the packet must also be updated—hence the need for an implementation of CRC32c. Another consideration was revealed during discussion of this feature with upstream developers. In typical ethernet networks, when a switch receives a packet that has an invalid checksum, the expected behaviour is to forward the packet normally. The end host will recognise that the packet is corrupt and deal with it accordingly. However, consider the case where a set-field action is used on a corrupt packet. To behave in the expected manner, this error in the checksum must be propagated across the field change. This is performed in the following manner: 1. Take a copy of the initial checksum from the packet 33 Chapter 5 Implementation 2. Calculate the correct checksum for the original packet 3. Modify a field in the packet 4. Calculate the new checksum for the modified packet 5. If the first two are not identical, apply the difference to the new checksum 6. Place the new checksum into the packet In the case where the original checksum is correct, the resulting checksum will be correct for the packet. Therefore, the destination host for the packet will see a valid packet. In the case where the original checksum was incorrect, the destination host will receive a packet with an invalid checksum. If the behaviour was not written in this manner, then it would be possible for an invalid packet to arrive at the destination host with a valid checksum. 5.3.1 End-to-End Testing A virtualised test environment was constructed to test the implementation of this feature. Figure 5.2 shows the test setup. The test PC is running Debian GNU/Linux 6.05, with the modified version of Open vSwitch connected to two Virtual Machines (VMs). Each of the VMs has Linux Kernel SCTP (lksctp) (IBM Corporation, 2001) installed for testing. One VM acts as a server, listening for SCTP traffic while the other is instructed to send SCTP packets to the other host. Virtual Machine Virtual Machine Open vSwitch Figure 5.2: SCTP test setup Chapter 5 Implementation 34 Initially, Open vSwitch acts as a standard ethernet switch—learning how to forward packets between the hosts, and allowing traffic to freely pass between them. A rule is installed into Open vSwitch to drop all SCTP traffic on the port being used. Subsequently, the SCTP traffic cannot reach the other host. Another application is used to test reachability between the two hosts, and indicates that the hosts can still communicate normally. Finally, the flow entry is deleted from Open vSwitch and the SCTP connection is observed to re-establish. The first attempt at setting up this test environment was made using Linux Containers (LXC). LXC is a lightweight virtualisation software that provides isolated Linux systems based on the host operating system (Lezcano, 2008). This minimises the set up required to run the containers. Additionally, the virtual environment will have access to the same software that is installed on the host. LXC provides a kernel module that allows the containers to have a separate process and network space. Unfortunately, despite the support for SCTP in the rest of the Linux networking stack, this module does not support SCTP. After a query regarding this behaviour on the lxc-users mailing-list, the developers confirmed this lack of support (Lezcano, 2012). For the second attempt, a more established virtualiser was chosen: QEMU (Bellard, 2003). Libvirt (Red Hat, Inc, 2005) and Kernel-based Virtual Machine (KVM) (Red Hat, Inc, 2007) were also used to speed up the installation and management of the VMs. This process involved installing a copy of Linux into each VM and installing lksctp. With these VMs set up, the test was carried out for the userspace datapath and the kernelspace datapath. This testing provided confirmation that the SCTP port matching was operating correctly. Before spending more time on implementing and writing automated tests for the remaining functionality in SCTP support, the patchset for this feature was submitted to the openvswitch-dev mailing-list. This was planned to give assurance that the approach was correct, particularly with the implementation of the kernel datapath code. This review process was delayed due to the acquisition of Nicira (Herrod, 2012), and was not carried out in time for this feature to be completed to the expected quality. The remaining time of this project was spent working on the OpenFlow Testing Framework and writing this report. 35 Chapter 5 Implementation 5.4 Evaluation Each prior section in this chapter has described the uses, implementation and testing of a feature individually. This section describes how well the implementation lines up with the goals set at the outset of this project. 5.4.1 Code Review The primary goal for this project was for all implementation to be accepted upstream. As of publication, eleven patches have been accepted into the Open vSwitch codebase. Table 5.1 shows a summary of the development over the course of this project. The development time includes time spent learning the tools and processes, but does not include any time spent after the first submission of that patchset. The review time is counted from the first submission of the patch until it is submitted into the upstream repository. With the exception of SCTP and misc patches, the implementation phase was usually carried out after the review process of the previous feature. Time not accounted for in the table includes time spent on testing using the OpenFlow Testing Framework. The notable delay in the review process for write-metadata is due to two factors: the patch was based against another patchset from the community which needed to go through the review process; and the acquisition of Nicira further delayed this process. This delay also affected the work on SCTP support. At the time of writing, the SCTP support is yet to be accepted upstream. The patch has received one round of review to date. However, due to time constraints, this round of feedback could not be integrated into the implementation. The number of reviews conducted for the first three features appears to increase each time. The first round for each usually solicited a discussion on the Feature Arbitrary ethernet Metadata matching Metadata writing SCTP Misc Patches Reviews 2 1 2 5 5 2 3 4 1 1ea Development 7 3 5 9 weeks weeks weeks weeks - Table 5.1: Development breakdown Review Time 2 weeks 1 week 14 weeks - Chapter 5 Implementation 36 implementation approach for the feature or an explanation of the architecture of Open vSwitch. In the case of metadata matching, one of the reviews was a simple request to remove an unused function which was unintentionally included in the patch. Metadata writing had the most reviews to date, in part due to the discussion on how the feature should be represented, and also due to the decision made to base the patch on an alternative branch of the codebase. In retrospect, this decision slowed development and provided additional workload for the developers involved. The recommendation from this experience is to use alternative branches of the codebase only in cases where branch is likely to be integrated into the main development branch before completion of the feature. 5.4.2 Testing features Each of the features that was accepted upstream included unit-tests with the code submission. These tests demonstrated the compliance with the OpenFlow protocol. This is reinforced by the reviews conducted by upstream developers, most of whom have had some involvement in the creation of the OpenFlow specifications. An additional guarantee of code compliance was also investigated—the use of the OpenFlow Testing Framework. However, several issues impeded the use of this software. These consist of problems running the framework, and problems with the representation of OpenFlow messages. Despite the goal for the testing framework to be easy to set up and use, there were particular hurdles faced in this area. Two versions of the framework were used: One from Ericsson Research, and one from CPqD. The latter of these required an additional library that is not developed primarily for Linux. There was some work involved in patching this codebase to have it build correctly on the test PC. Even when the framework was successfully installed, it exhibited unusual behaviour; the failure of one test would prevent the next test from running correctly. This behaviour was traced to the way in which it reserved addresses on the local host for sending OpenFlow messages. Furthermore, despite the support of various OpenFlow 1.1 features in Open vSwitch, the current behaviour is to only use these through Nicira extensions. As of publication, the ofproto component that brokers connections with Chapter 5 Implementation 37 OpenFlow controllers will not negotiate an OpenFlow 1.1 connection. A patchset to add this feature is currently undergoing review on the mailinglist. However, due to time constraints, this patchset could not be used to test the functionality of this project. Chapter 6 Conclusion This report has documented the process of learning about a large open source project and contributing features to the codebase. This process involved a cycle of learning about the project and its architecture, discussing how modifications should be made, implementing those modifications, and the steps involved in ultimately getting the code into the upstream codebase. An exploration of the computer networking concepts was presented, to provide a basis for this project—Software-Defined Networks, OpenFlow, and how Open vSwitch implements these. This was expanded upon with specific reference to the OpenFlow specifications and Open vSwitch design documentation. The use of OpenFlow features such as instructions, experimenter extensions and extensible matches was described and linked to the architecture of Open vSwitch. These elements—the concepts, process, and existing architecture—provided a basis for the implementation of arbitrary ethernet masking, metadata support, and SCTP support. The details of each of these features as explained and linked to how they were implemented in the Open vSwitch codebase. Finally, the implementation of each feature was assessed programmatically through software tests and independently by upstream developers. This resulted in the acceptance of eleven patches into the mainline codebase. 6.1 Impact This project resulted in the addition of new OpenFlow features and general improvements to the Open vSwitch codebase. With the completion of the review process for SCTP support, this will also involve the inclusion of code Chapter 6 Conclusion 39 into the Linux kernel. This work provides additional flexibility to researchers seeking to use a production-quality software switch in their experiments. In terms of its positioning as part of Project W, this project pushes one of the LSR components closer to compatibility with OpenFlow 1.1. This version of the protocol standardised extensions used by Kempf et al. for the original OpenFlow-based LSR platform. The contributions of this project are step towards providing a more accessible open source LSR. 6.2 Future Work Implementing features for OpenFlow 1.1 and above in Open vSwitch is an active development area. While the majority of work towards OpenFlow 1.1 support in Open vSwitch has been contributed, there are some remaining features. To provide an interoperable platform with support for a wide range of features, support for the later 1.2 and 1.3 protocols will also need to be developed. These developments would provide the research community with a fully OpenFlow-compliant datapath. Further work will need to be carried out in the broader community to provide support for these versions of OpenFlow in controller software. References Bellard, F. (2003). QEMU. http://www.qemu.org/. Retrieved 20 October, 2012, from Bianco, A., Birke, R., Giraudo, L., Palacin, M. (2010). OpenFlow Switching: Data Plane Performance. In ICC, pp. 1–5. IEEE. Big Switch Networks (2011). Indigo - Open Source OpenFlow Switches. Retrieved 8 October, 2012, from http://www.openflowhub.org/display/Indigo. Big Switch Networks (2012). OFTest—Validating OpenFlow Switches. Retrieved 8 October, 2012, from http://oftest.openflowhub.org/. Bird, C., Nagappan, N. (2012). Who? Where? What? Examining distributed development in two large open source projects. In Lanza, M., Penta, M. D., Xi, T. (Eds.), MSR, pp. 237–246. IEEE. Calhoun, P. R., Loughney, J., Arkko, J., Guttman, E., Zorn, G. (2003). Diameter Base Protocol. RFC 3588. Calleja, D. (2012). Linux 3.3. http://kernelnewbies.org/Linux 3.3. Retrieved 22 October, 2012, from CPqD (2012). OpenFlow Software Switch. Retrieved 8 October, 2012, from http://github.com/CPqD/. Donnet, B., Bonaventure, O. (2008). On BGP communities. SIGCOMM Comput. Commun. Rev., 38 (2), 55–59. doi:10.1145/1355734.1355743. Ericsson Research (2011a). OFTest for OpenFlow 1.1. Retrieved 8 October, 2012, from http://github.com/TrafficLab/oftest11. Ericsson Research (2011b). OpenFlow 1.1 Software Switch. Retrieved 8 October, 2012, from http://github.com/TrafficLab/of11softswitch. 41 References Erlang Solutions (2012). FlowForwarding/LINC-Switch. Retrieved 8 October, 2012, from http://github.com/FlowForwarding/LINC-Switch. Herrod, S. (2012). VMware and Nicira—Advancing the Software- Defined Datacenter. Retrieved 23 http://blogs.vmware.com/console/2012/07/. September, 2012, from Hölzle, U. (2012). OpenFlow @ Google. Open Networking Summit. IBM Corporation (2001). LKSCTP. http://lksctp.sourceforge.net/. Retrieved 20 October, 2012, from IEEE Computer Society (2002). IEEE Standard for Local and Metropolitan Area Networks: Overview and Architecture. New York, NY: IEEE. Kempf, J., Whyte, S., Ellithorpe, J., Kazemian, P., Haitjema, M., Beheshti, N., Stuart, S., Green, H. (2011). OpenFlow MPLS and the open source label switched router. In Proceedings of the 23rd International Teletraffic Congress, ITC ’11, pp. 8–14. ITCP. Lezcano, D. (2008). lxc Linux Containers. Retrieved 20 October, 2012, from http://lxc.sourceforge.net/. Lezcano, network D. (2012). protocols. Re: Retrieved 20 [Lxc-users] Alternative October, 2012, from http://www.mail-archive.com/[email protected]/msg03826.html. McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., Turner, J. (2008). OpenFlow: enabling innovation in campus networks. SIGCOMM Comput. Commun. Rev., 38 (2), 69–74. Open Networking Foundation (2012). The OpenFlow Switch Specification (1.2 and above). Retrieved 22 March, 2012, from http://www.opennetworking.org/about/onf-documents. Open vSwitch (2012a). Open vSwitch Documentation. Retrieved 20 October, 2012, from http://openvswitch.org/support/. Open vSwitch (2012b). Open vSwitch Mailing Lists. Retrieved 20 October, 2012, from http://openvswitch.org/mlists/. 42 References OpenFlow.org (2011). The OpenFlow Switch Specification (1.0,1.1). Retrieved 22 October, 2012, from http://www.openflow.org/wp/documents/. Pfaff, B. (2011). Call for assistance: OpenFlow 1.1 and 1.2 support in Open vSwitch. Retrieved 21 March, 2012, http://www.mail-archive.com/[email protected]/msg06532.html. Pfaff, B. (2012a). [PATCH v2 00/11] instruction actions/goto-table support. Retrieved 20 October, 2012, http://www.mail-archive.com/[email protected]/msg11500.html. from applyfrom Pfaff, B. (2012b). [PATCH v2] ofp-actions: Implement writing to metadata field. Retrieved 20 October, 2012, from http://www.mail-archive.com/[email protected]/msg11290.html. Pfaff, B., Pettit, J., Koponen, T., Amidon, K., Casado, M., Shenker, S. (2009). Extending networking into the virtualization layer. In HotNets-VIII. Red Hat, Inc (2005). Libvirt: The virtualization API. Retrieved 20 October, 2012, from http://libvirt.org/. Red Hat, Inc (2007). Kernel Based Virtual Machine. Retrieved 20 October, 2012, from http://www.linux-kvm.org/. Riel, R. (2006). UpstreamMerge/SubmitChecklist. Retrieved 22 October, 2012, from http://kernelnewbies.org/UpstreamMerge/SubmitChecklist. Stanford University (2012). Clean Slate Design for the Internet. Retreived 12 October, 2012, from http://cleanslate.stanford.edu/. Stewart, R. R. (2007). Stream Control Transmission Protocol. RFC 4960. Traina, P., Chandrasekeran, R., Li, T. (1996). BGP Communities Attribute. RFC 1997. Glossary action How the forwarding element should handle particular matches . AutoTest Allows developers to create platform-independent test cases. Part of GNU Build tools . BGP Border Gateway Protocol. The de facto standard protocol for sharing routes with other organisations . black-box Testing the behaviour of a component based on sending particular inputs to the component, and monitoring the outputs . controller The part of a forwarding element which is responsible for route aggregation and advertisement . CRC32c The checksum algorithm used in SCTP to detect transmission or storage errors for the data contained in a packet . datapath The part of a forwarding element which is responsible for the forwarding of packets. Typically used to refer to a software or hardware switch . flow A class of packets, as determined through a set of common attributes such as the same source or destination address. . flow entry A combination of a match and action . forwarding element A router or switch . Forwarding Information Base A table which stores forwarding entries for fast packet switching . Host Byte Order The endianness of bytes when the data is stored in memory. Architecture dependent . Glossary 44 Label-Switched Router A type of router that provides label-switching functionality. This functionality is used to simplify traffic management in hightraffic network environments. . match The classification of a packet based on particular field values . Network Byte Order The endianness of bytes when the data is written to the network interface. Typically standardised as big-endian—Most significant bytes first . Sparse The semantic parser, a compiler frontend and static code analyzer for ANSI C programs . white-box Testing the behaviour of a component with reference to the component’s structure, often by using shortcut code that allows direct access to internal functions .
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project