Software

VNET: An Ethernet Layer Adaptive Virtual Network

VNET was developed as part of the Virtuoso Project run by Prof. Peter A. Dinda at the Department of Electrical Engineering and Computer Science, Northwestern University.
Description

Virtuoso is a middleware system for virtual machine distributed computing that for a user very closely emulates the existing process of buying, configuring, and using an Intel-based computer or collection of computers from a web site, a process with which many users and certainly all system administrators are familiar.

The virtual machines purchased by an user through Virtuoso, could be hosted on multiple sites, with different network management and security philosophies, often spread over the wide area. Running a virtual machine on a remote site is equivalent to visiting the site and connecting a new machine. The network presence the virtual machine gets is completely dependent on the foreign site. This situation is further complicated as the number of sites is increased and if we permit virtual machines to migrate from site to site.

VNET is the part of our system that creates and maintains the networking illusion, that the user's virtual machines are on the user's local area network. It is a simple link layer proxying scheme that works entirely at user level. Using VNET, virtual machines have no network presence at all on a remote site. Instead, VNET provides a mechanism to project their virtual network cards onto another network (the user's home network), which also moves the network management problem from one network to another. Because the virtual network is a link layer network, a machine can be migrated from site to site without changing its presence, it always keeps the same IP address, routes, etc.

Top

Operation

Each physical machine that can instantiate virtual machines (a host) runs a single VNET daemon. One machine on the user's network also runs a VNET daemon. This machine is referred to as the Proxy. Figure 1 shows a typical startup configuration of VNET for four hosts, each of which may support multiple VMs. Each of the VNET daemons running on the foreign hosts is connected by a TCP connection (a VNET link) to the VNET daemon running on the Proxy. We refer to this as the resilient star backbone centered on the Proxy. By resilient, we mean it will always be possible to at least make these connections and re-establish them on failure.

Top

The VNET daemons running on the hosts and Proxy open their virtual interfaces in promiscuous mode using Berkeley packet filters. Each VNET daemon has a forwarding table, Figure 2 shows one such forwarding table at a VNET daemon. Each packet captured from the interface or received on a TCP connection is matched against this forwarding table to determine where to send it, the possible choices being sending it over one of its outgoing links (TCP / UDP) or writing it out to one of its local interfaces using libnet, which is built on packet sockets, available on both Unix and Windows. If the packet does not match any rule then no action is taken. For each packet multiple rules might be matched at the same time. Each match has a priority value associated with it, calculated dynamically based on the strength of the match. The stronger thematch, the higher the priority value. For example, a rule matched with the "any" qualifier will have a lower priority than a rule matched with exact values. The rule with the highest priority is used. The rule that has the destination address as "none" is the default rule. This rule is always matched as long as the source addresses match, but has the lowest possible priority. If only the default rule is matched then the packet would be sent over the TCP connection to the VNET daemon on the Proxy.

Top

Figure 3 helps to illustrate the operation of a VNET link. Each successfully matched packet is also passed to the Virtual Topology and Traffic Inference Framework, VTTIF, to determine the local traffic matrix. Each VNET daemon periodically sends its inferred local traffic matrix to the VNET daemon on the Proxy. The Proxy, through its physical interface, provides a network presence for all the VMs on the user's LAN and makes their configuration a responsibility of the user. The first generation of VNET was limited solely to this star topology, thus all traffic among the users' VMs would be forwarded through the central Proxy, resulting in extra latency and a bandwidth bottleneck. The star would be used regardless of the application, as its sole goal was to provide connectivity for the VMs regardless of the security constraints on the various sites. The second generation VNET removes this restriction. Now, the star topology is simply the initial configuration, again to provide connectivity for the VMs. Additional links and forwarding rules can be added or removed at any time. This makes topology adaptation, as we describe in this thesis, possible. Figure 4 shows a VNET configuration that has been dynamically adapted. Also the links can either use TCP or UDP.

Top

Primitives
A VNET client can connect to any of the VNET daemons to query status or perform an action. Following are the primitives made available by VNET. The primitives generally execute in about 20 ms, including client time. On initial startup VNET can calculate an upper bound on the time taken to configure itself (or change topology).

Top

Grammar and VNET Tools

Building on the primitives, we have developed a language for describing the topology and forwarding rules. Figure 5 defines the grammar for the language. The tools we use here take the form of scripts that generate or parse descriptions in that language. These tools provide functionality such as:

Top

Performance

The overhead of using VNET is mitigated over the wide area. In a 100 Mbps Ethernet LAN, VNET is as fast as state of the art network virtualization software, which in turn is almost as fast as the native hardware. However, over faster networks, such as optical networks operating at Giga-bit speeds or higher, VNET is a factor of two slower than commercial virtual network solutions, which in turn are a factor of 3 slower than the native hardware. We are currently working on improving the performance of VNET on faster networks

Top

Publications
VNET is described in the following papers:

A. Sundararaj, A. Gupta, P. Dinda, "Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation", Proceedings of the Fourteenth IEEE International Symposium on High Performance Distributed Computing (HPDC-14), July 2005. 

A. Sundararaj, A. Gupta, P. Dinda, "Dynamic Topology Adaptation of Virtual Networks of Virtual Machines", Proceedings of the Seventh Workshop on Langauges, Compilers and Run-time Support for Scalable Systems (LCR 2004), October 2004. 

A. Sundararaj, P. Dinda, "Towards Virtual Networks for Virtual Machine Grid Computing", Proceedings of the third USENIX Conference on Virtual Machine Technology (VM 04), May 2004. An earlier version is available as Technical Report NWU-CS-03-27  

Top

Talks
VNET is described in the following presentations:

A. Sundararaj, "Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation", Fourteenth IEEE International Symposium on High Performance Distributed Computing (HPDC-14), July 2005, Research Triangle, NC. 

A. Sundararaj, "Dynamic Topology Adaptation of Virtual Networks of Virtual Machines", Proceedings of the Seventh Workshop on Langauges, Compilers and Run-time Support for Scalable Systems (LCR 2004), October 2004, Houston, TX. 

A. Sundararaj, "Towards Virtual Networks for Virtual Machine Grid Computing", Proceedings of the third USENIX Conference on Virtual Machine Technology (VM 04), May 2004, San Jose, CA. 

Top

Source Code
Authors
Ananth I. Sundararaj
Peter A. Dinda

Top



Ananth I. Sundararaj
Last modified: Tue Oct 25 22:18:28 CST 2005