A) openvswitch

*** new entry on policy routing for 2 networks and NIC cards go here to check it out ******

Originally I titled this page, “The 3 bridges”, then I changed it to ” bridging – openvswitch, KVM and Linux and Bridge” Finally I shorted it to Software Defined Networking. But its mostly on the 3 bridges. I really only care about openvswitch. The other 2 bridges really just get in the way. Although I keep the bridge-utils around for compatibility. Although I should probably do away them.

This is the process I used and modified for getting Openvswitch working. Openvswitch is part of the openstack environment, but for testing purposes, this was set up on a stand alone server without any Openstack components just KVM.

1) Start with the typical update/upgrade thing; apt-get update && apt-get dist-upgrade
2) Then we install kvm and the libvirt stuff; apt-get install kvm libvirt-bin virtinst

Now we have a bit of a problem here. See bridge-utils is gonna get installed too as it gets installed in precise-pagolin with libvirt-bin (Bridge #1 – Linux bridge) so when you reboot it sets up the linux bridge and when you try to startup openvswitch with “service openvswitch-switch start” we are going to get:

root@openvswitch:/usr/src/openvswitch-1.4.0# service openvswitch-switch start
* bridge module is loaded, not loading brcompat
Module has probably not been built for this kernel.
For instructions, read
/usr/share/doc/openvswitch-datapath-source/README.Debian
* bridge module is loaded, not loading brcompat

We’ll deal with this later, we are going to leave this like this for now to illustrate the error you will get for not dealing with this now….

3) So now install openvswitch et al, apt-get install openvswitch-controller openvswitch-brcompat openvswitch-switch openvswitch-datapath-source

now everything is installed. we blow away bridge #2 to avoid any conflicts; this bridge comes with KVM – virbr0 (Bridge #2 – virbr0 KVM default bridge)

Want to check that it’s there?

virsh net-list
Name State Autostart
—————————————–
default active yes

ok, so let’s blow it away….

# virsh net-destroy default
# virsh net-autostart –disable default
4) then we purge ebtables

aptitude purge ebtables
5) then we do the compat thing

BRCOMPAT=yes in /etc/deafault/openvswitch-switch
6) then we do that start thing and it fails…..

service openvswitch-switch start
root@openvswitch:/usr/src/openvswitch-1.4.0# service openvswitch-switch start
* bridge module is loaded, not loading brcompat
Module has probably not been built for this kernel.
For instructions, read
/usr/share/doc/openvswitch-datapath-source/README.Debian
* bridge module is loaded, not loading brcompat

7) now if you do what they say…. by looking at /usr/share/doc/openvswitch-datapath-source/README.Debian

it tells you to do this…..

root@openvswitch:~# more /usr/share/doc/openvswitch-datapath-source/README.Debian

Open vSwitch for Debian

———————-
* How do I build this module the Debian way?
– Building with module-assistant:
$ module-assistant auto-install openvswitch-datapath
or
$ m-a a-i openvswitch-datapath
If kernel source or headers are in a non-standard directory, add
the option -k /path/to/kernel/source with the correct path.
– Building with make-kpkg
$ cd /usr/src/
$ tar jxvf openvswitch.tar.bz2
$ cd /usr/src/kernel-source-2.6.26
$ make-kpkg –added-modules=openvswitch modules
– Building without make-kpkg
$ cd /usr/src/
$ tar jxvf openvswitch.tar.bz2
$ cd modules/openvswitch
$ fakeroot debian/rules kdist_image
If you run this as root, fakeroot is not needed.

— Ben Pfaff , Wed, 22 Jun 2011 09:51:28 -0700

root@openvswitch:~#

so …. we do that and it still fails….after we issue the suggest module-assistant auto-install command (which was loaded previously but..)

root@openvswitch:/usr/src/openvswitch-1.4.0# module-assistant auto-install openvswitch-datapath

Updated infos about 1 packages
Getting source for kernel version: 3.2.0-52-generic
Kernel headers available in /usr/src/linux
Creating symlink…
Couldn’t create the /usr/src/linux symlink!
apt-get install build-essential
Reading package lists… Done
Building dependency tree
Reading state information… Done
build-essential is already the newest version.

0 upgraded, 0 newly installed, 0 to remove and 24 not upgraded.
Done!
unpack
Extracting the package tarball, /usr/src/openvswitch-datapath.tar.bz2, please wait…
“/usr/share/modass/packages/default.sh” build KVERS=3.2.0-52-generic KSRC=/usr/src/linux-headers-3.2.0-52-generic KDREV=3.2.0-52.78 kdist_image
Done with /usr/src/openvswitch-datapath-module-3.2.0-52-generic_1.4.0-1ubuntu1.5_amd64.deb .
dpkg -Ei /usr/src/openvswitch-datapath-module-3.2.0-52-generic_1.4.0-1ubuntu1.5_amd64.deb

Version 1.4.0-1ubuntu1.5 of openvswitch-datapath-module-3.2.0-52-generic already installed, skipping.

Here’s the part we care about, after completely this step successfully, we issue a status and. well not good.

root@openvswitch:/usr/src/openvswitch-1.4.0# service openvswitch-switch status
ovsdb-server is not running
ovs-vswitchd is not running
ovs-brcompatd is not running

If we try to start it, it says the bridge module is loaded so we are back where we started….

root@openvswitch:/usr/src/openvswitch-1.4.0# service openvswitch-switch start
* bridge module is loaded, not loading brcompat
Module has probably not been built for this kernel.
For instructions, read
/usr/share/doc/openvswitch-datapath-source/README.Debian
* bridge module is loaded, not loading brcompat

so….now what…

well the problem is the unix bridge that gets installed with bridge-utils, which gets installed by libvirt-bin in precise-pagolin sets up the linux bridge which is going to get installed during the install is in the way, you need to nuke the unix bridge so openvswitch can do its thing so…

run this

7) rmmod bridge – which gets rid of the bridge

8) now lets start the openvswitch

first check the status….

root@openvswitch:~# service openvswitch-switch status
ovsdb-server is not running
ovs-vswitchd is not running
ovs-brcompatd is not running

so let’s start it up …

root@openvswitch:~# service openvswitch-switch start
* Inserting brcompat module
* Starting ovsdb-server
* Configuring Open vSwitch system IDs
* Starting ovs-vswitchd
* Starting ovs-brcompatd
* Enabling gre with iptables

then check the status…

root@openvswitch:~# service openvswitch-switch status

ovsdb-server is running with pid 10522
ovs-vswitchd is running with pid 10531
ovs-brcompatd is running with pid 10572

9) some commands

root@openvswitch:~/openvswitch# lsmod | grep brcom

brcompat 13512 0
openvswitch 84038 2 brcompat

root@openvswitch:~/openvswitch# ovs-vsctl show

43fb1d86-a3f0-49a4-96a2-0026db6dd0f3
Bridge “br0”
Port “eth0”
Interface “eth0”
Port “br0”
Interface “br0”
type: internal
ovs_version: “1.4.0+build0”

root@openvswitch:~/openvswitch# ovs-ofctl show br0

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000b8ac6f830247
n_tables:255, n_buffers:256
features: capabilities:0xc7, actions:0xfff
1(eth0): addr:b8:ac:6f:83:02:47

config: 0
state: 0
current: 100MB-FD COPPER AUTO_NEG
advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-HD 1GB-FD COPPER AUTO_NEG AUTO_PAUSE
supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-HD 1GB-FD COPPER AUTO_NEG

LOCAL(br0): addr:b8:ac:6f:83:02:47

config: 0
state: 0

OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0

root@openvswitch:~/openvswitch# ovs-dpctl show

system@br0:
lookups: hit:41614 missed:7589 lost:0
flows: 4
port 0: br0 (internal)
port 1: eth0

root@openvswitch:/usr/src/openvswitch-1.4.0# dpkg -l | grep openv

ii openvswitch-brcompat 1.4.0-1ubuntu1.5 Open vSwitch bridge compatibility support
ii openvswitch-common 1.4.0-1ubuntu1.5 Open vSwitch common components
ii openvswitch-controller 1.4.0-1ubuntu1.5 Open vSwitch controller implementation
ii openvswitch-datapath-dkms 1.4.0-1ubuntu1.5 Open vSwitch datapath module source – DKMS version
ii openvswitch-datapath-module-3.2.0-52-generic 1.4.0-1ubuntu1.5 Open vSwitch Linux datapath kernel module
ii openvswitch-datapath-source 1.4.0-1ubuntu1.5 Open vSwitch datapath module source – module-assistant version
ii openvswitch-pki 1.4.0-1ubuntu1.5 Open vSwitch public key infrastructure dependency package
ii openvswitch-switch 1.4.0-1ubuntu1.5 Open vSwitch switch implementations

Then we build a ubntu 12.04 VM, and change xml file for that VM to point to the bridge and

if we didn’t change the bridge to the openvswitch bridge, but did change the VM’s XML file then we’d get the following:

root@openvswitch:~# virsh list –all
Id Name State
—————————————————-
– ubuntu1204 shut off

root@openvswitch:~# virsh edit ubuntu1204
Domain ubuntu1204 XML configuration edited. (here’s where edited the xml file)

Here’s what was editted during the virsh edit ubuntu1204….

VMxml

root@openvswitch:~# virsh start ubuntu1204
error: Failed to start domain ubuntu1204
error: Unable to add port vnet0 to OVS bridge br0: Operation not permitted <=== this failed because we had the old linux bridge installed after a reboot, so run the scripts to remove the linux bridge and old the opencswitch bridge, and then start the VM again…:

============================
root@openvswitch:~/openvswitch# ovs-vsctl show
43fb1d86-a3f0-49a4-96a2-0026db6dd0f3
Bridge "virbr0"
Port "virbr0"
Interface "virbr0"
type: internal
Bridge "br0"
Port "eth0"
Interface "eth0"
Port "vnet0"
Interface "vnet0"
Port "br0"
Interface "br0"
type: internal
ovs_version: "1.4.0+build0"
root@openvswitch:~/openvswitch#

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s