My 1st lab I'm blogging shows how you can setup a simple virtual lab to create VXLAN's on then use wireshark to view the VXLAN traffic and peek inside the VXLAN traffic by decoding the packets.
On my lab I have a single VM for vCenter 5.1, 2 ESXi 5.1 hosts and a shared storage virtual appliance.
The vCenter and each host as well as storage appliance has a vNic in the VMNet1(host only) Workstation network. Each host also has a vNic in the VMNet4(host only) Workstation network that is DHCP enabled for VXLAN transport. My lasptop has the IP address 192.168.142.1 and has vSphere 5.1 client and Wireshark installed. I have created a single DC with one cluster that I have deployed the vShield Manager "VSM" into. I have also created two VM's for testing. I've used OpenBSD as they have a small footprint and can run various services for testing network traffic.
Please note figure 1.0 in Part two of this blog post on how the lab was built on VMware Workstation.
Once the VSM has booted up and had its IP, Subnet, default gateway and DNS configure from the VSM CLI I registered it with my vCenter Server.
Once the VSM plugin has registered with vCenter Client you navigate to the DC and click the "Network Virtualization" tab. Then click on preperation and edit for the segment ID. Enter some test values in this case a multicast group range of 224.1.1.50-224.1.1.60 and segment ID pool of 5000-5100.
Once the segment ID is added click on connectivity>edit and select the cluster that you want to deploy VXLAN over. This assumes I have already created the vNetwork Distributed switch version 5.1 accross all hosts in the cluster.
At this point VXLAN will be deployed onto each host in the cluster, a new VMkernel interface for VXLAN will be created on each hosts in my case vmk1. The vmk1 interfaces will be assigned an IP address using DHCP from the VMware workstation VMNet4 IP address pool.
At this point the Cluster is prepared for VXLAN and should have a status of Normal.
We now create the Scope for the VXLAN's in this case a single cluster. Select add then give the scope a name, descipton and select the cluster(s) for this scope.
Great, VXLAN is now deployed and ready for us to create our 1st VXLAN. Each VXLAN wil be mapped to a multicast group, if we assigned just two multicast addresses to our range and created 3 VXLANs you would have 2 virtual wires mapped to a single multicast group. This is not a problem as far as I know but you would need to be mindful of mutliple VXLANs broadcast domains being on the same multicast tree and would make pruning each broadcast domain back harder due to a single multicast group for multiple VXLAN's so might need carefull thought and planning.
So now to create our 1st VXLAN, simply click on Networks then the green cross for add. Give the VXLAN a name, description and scope which in this case has to be scope-01. Then click ok. The VXLAN is then created and an associated Portgroup is created. The VXLAN is given a VNI "VXLAN Network Identifier" of 5000 and mapped to the multicast group 224.1.1.50.
Now we will add each of our two OpenBSD VM's into the VXLAN,
Now run a continous ping from VM01 "172.16.1.2" to VM02 "172.16.1.3". Then open the packet capture tool Wireshark and choose the interface for VMNet4 which is the virtual network on Workstation we are transporting the VXLAN traffic over.
Now start the packet capture whilst the ping session is still running from VM01 to VM02. It is important to make sure that VM01 and VM02 are NOT running on the same host or you will be sat like I was wondering why I didnt see any traffic. The reason for this is we want to see the VXLAN traffic going between hosts over the VXLAN transport network.
When the packet capture starts you should see the VXLAN UDP traffic with a source IP of one ESXi host and a destination of the other ESXi host, with a destination UDP port of 8472. Looking at the packet capture we can only see that it is a VXLAN packet, and that it has a payload. The important thing here is that the traffic is unicast what we havent done is captured the traffic that transported the ARP request using multicast, we will do this later in this blog.
What we can now do is use the VXLAN decoder in Wireshark to look agt the VXLAN header and the contents of the packet. To do this select a packet and right click then select "decode as" and choose a transport protocol of VXLAN.
Now we should see the contents of the VXLAN payload i.e. the original packet which in this case is the ICMP echo-request and echo-reply. You can also view the VXLAN header showing the VNI of 5000 which is the VXLAN Network Identifier which was assigned to our VXLAN.
As we missed the original ARP request sent by VM01 for VM02's MAC address from VM01 we will run a continous ping to an IP address not in use 172.16.1.4. this will keep VM01 sending an ARP request out as a broadcast. VXLAN will then encapsulate this packet into a multicast packet onto the multicast group 224.1.1.50 and this will be sent to all hosts that would have joined this group through IGMP. In our case we are not using PIM or IGMP snooping. As you can see in the none-decoded traffic flow below the source remains the hosts IP address but the destination is now the multicast group for this and possibly other VXLAN's.
In part 2 of this blog post we will decode the VXLAN multicast traffic, and show what information you can view on the ESXi hosts via an SSH session.
Kevin Barrass