Holy Switch! 10GbE Switching in the Home Lab

XS708EOver the last five to six months I set out to revamp/rebuild both the compute nodes and networking layer in my home lab. These lab upgrades seem to pop up post VMworld (posts here and here) as the idea of running the latest and greatest software from either VMware or its partner eco-system gets my mind racing and the PayPal account to open up.

With this latest round of host upgrades, I was able to introduce 10GbE networking functionality into the lab as the new host servers have dual integrated Intel X540 10GbE adapters. That put me into the market for 10GbE switch as my current switch (Cisco 2960) only supports 1GbE links. Jumping over to NewEgg.com I took a quick look at the available options for 10GbE switching and as you can imagine there are not a lot of “affordable” options out there. The main two models that jumped out where both made by Netgear, the Pro-Safe XS712T (12 ports) and the Pro-Safe XS708E (8 ports).

In comparing the two models I end up selecting the XS708E 8 port model for both a cost reason (the price per port was cheaper) and that I would either need to add two more hosts or a 10GbE Synology array in the future to really tap into the extra ports provided(I keep telling myself this was good reasoning Smile ). As it stands now, with my three hosts I can dedicate one 10GbE interface for VSAN traffic and the other for vMotion traffic, and have two ports free on the switch for future expansion.

Switch Features

While the overall feature I was looking for was the need for speed, the NetGear Pro-Safe XS708E does some additional things to the table:

  • VLAN Support – An obvious need to allow for network segmentation. Created two VLAN’s in the lab, one for VSAN and the other for vMotion traffic as mentioned above.
  • Quality of Service (QoS) – Tagging of traffic for network prioritization
  • Loop Prevention – Loops = BAD
  • IGMP Snooping – Provides for the listening of IGMP network traffic. Requirement for VMware VSAN  
  • Rate Limiting – Allows for network bandwidth throttling on a given switch port
  • Link Aggregation (LAG) – Ability to bond two ports together to aggregate bandwidth
  • Jumbo Frames – Support for larger Ethernet frames to increase payload size (from 1500 to 9000 bytes)

Speaking of Speeds

The overall setup and implementation of the switch is extremely straightforward so I decided not to cover the topic or do a section on the process. What I thought I would do is just jump straight to the speed and feeds. For the performance testing the lab environment was configured as follows:

  • Two Tiny Core Linux virtual machines were used as the “Client” and “Server”
  • IPerf v3 was used as the performance tool
  • Each Tiny Core Linux virtual machine resided on a separate host
  • Virtual machine networking consisted of a single Standard Virtual Switch with a 10GbE uplink

Below is the output from one of the recent IPerf test runs. I am constantly getting speeds of 5 Gbits/second. With more additional tweaking and testing I am hoping to get that number higher.

IPerf

And for the less scientific test, aka sit of the pants testing, I conducted a vMotion test across both a single 1GbE link and a single 10GbE link. The virtual machine was configured with 8GB of RAM and I ran each configuration twice (with the exact same results). As seen in the screen shot below, the section highlighted in RED is the 10GbE testing. Each vMotion task completed in 18 seconds. The section highlighted in PURPLE is the configuration changes made on each of the hosts to go from vMotion running on the 10GbE uplinks over to the 1GbE uplinks. And finally, the section highlighted in GREEN displays the same virtual machine being migrated leveraging the 1GbE uplinks. As you can see, each test completed in 27 seconds, or basically 9 seconds slower than the 10GbE tests:

Tasks

Final Thoughts

With the main goal achieved of introducing 10GbE connectivity into the lab I am happy with the NetGear switch and the numbers I am seeing out of it. A few additional mentions below:

Pros:

  • Price – Compared to anything else out there, the Netgear Pro-Gear by far as the cheapest buy in price
  • Great feature set to tune/play with in the lab
  • Ease of setup. You will be up and running in very short amount of time

Cons:

  • Device Management – While the switch is easy to setup and configure it requires the installation of the “Pro-Safe Plus Utility” IE fat client. No native support for HTML or CLI access/management
  • Noise – Though not something high on my list for those who have their labs in their office or another room in the house, the switch fans do produce a decent amount of noise

-Jason

%d bloggers like this: