Notes From the Field – Hybrid or All Flash VSAN?

tier-whatOver the last several months I have been involved in a few customer meetings where the customer has been looking to migrate or move away from a “traditional” storage architecture/array and the subject of Hyper Converged Infrastructure (HCI) seems to come up. Usually the topic covers “one of the big three” in the HCI space, be it Nutanix (XC for us Dell partners), Simplivity, or VMware’s VSAN product.  Recently though I have come across two projects where the customer initial started out looking at VMware VSAN in its Hybrid configuration (SSD for cache/buffer and magnetic disk for capacity) but then switched over to (or at least the conversation) of looking at the All Flash version and its corresponding data reduction technologies in VSAN 6.2 (Deduplication/Compression/Erasure Coding).

For these two scenarios (actual customer scenarios), I was interested in seeing if going All Flash could pretty much be the “standard” deployment model based on VSAN 6.2 enhancements or if cost could be still be a limiting factor. For the pricing models that follow, I used list pricing for both VMware and Dell for the software and hardware. I would not expect one to pay these prices on the street, they were used strictly for a cost comparison.

Scenario #1

For this project this was one of my first experience with working with/sizing an All-Flash VSAN configuration. When I originally met with the customer we were already going down the road of VSAN and the features/benefits it brought to the table for their environment. During the conversation we discussed potentially going down the All Flash route as VSAN 6.2 had recently been released. I wasn’t sure of the pricing, but figured if I was quoting/building a Hybrid configuration the level of effort of putting together an AF configuration as well wasn’t much higher.

For the requirements I won’t get too far into the details as it isn’t relevant to the blog post, but the sizing details is what I want to focus on. The customer was looking for a 50TB of usable storage capacity and 2TB of usable memory. Basing the initial build on a Hybrid configuration I knew I need to land around ~100TB of RAW capacity to meet the usable requirement based on VSAN Failures to Tolerate of one (FTT=1).

For the sizing of the AF VSAN configuration there was a little bit more effort involved with working the sizing numbers. With the introduction of data reduction features like dedupe and compression along with erasure coding (RAID5) to reduce VSAN overhead I knew I didn’t need to target a basic two to 1 ratio. In this scenario I factored in a conservative dedupe/compression of 3.5x reduction in data. VMware documentation mentions higher, but I wanted to play it safe.

With requirements and assumptions laid out, I went to work on a server design for each configuration. The table below provides a breakdown of the hardware and software licensing. Note – These are list prices and or prices I pulled from the Dell website, not actual numbers quoted. Your discounts may vary.

Attribute Hybrid Configuration All Flash Configuration
Server Vendor Dell Dell
Server Model R730XD FC430
CPU 2 x Intel E5-2630v3 2 x Intel E5-2630v3
Memory 8 x 32GB RDIMMS 8 x 32GB RDIMMS
Network 2 x Intel X520 Dual Port 10GB 2 x 10GB LAN on Motherboard
Caching Tier 2 x 800GB Mix Use SSD 2 x 800GB Write Intensive SSD
Capacity Tier 14 x 1TB 7.2K NL-SAS Drives 10 x 480GB Read Intensive SSD
RAID Controller PERC H730 FD332 Dual PERC
VSAN Licensing 16 x VMware VSAN 6 Standard 16 x VMware VSAN 6 Advance


With a rough idea of the hardware laid out. Let’s take a look at the pricing model for hardware and the required VMware VSAN software. These are list prices taken from both Dell’s and VMware’s websites.

Attribute Hybrid Configuration All Flash Configuration
List Price for Servers $200,992.00 $154,119.00
List Price for VSAN Licensing $39,920.00

(16 x $2,495.00)


(16 x $3,995.00)

Total Solution List Price $240,912.00 $218,039.00


Pretty good savings there as the All Flash configuration comes in at more than $22,000.00 less than the hybrid configuration. On top of that, we can pretty much say the AF configuration is going to provide a lot more performance for those dollars. But, for those paying closing attention, you will notice I didn’t do a “like for like” comparison on the hardware platform. In this scenario the customer chose to go with Dell’s FX2 converged product when looking at All Flash as opposed to staying with a traditional rack mount server, like the R730XD. So maybe that throws the numbers of some? Let’s find out.

Scenario #2

Ok, there might be some skeptics out there as I threw a bit of a curve ball in the first scenario with using different hardware platforms to ultimately get to the end goal. For the second scenario we get to really see if the numbers jive as we are doing as close a comparison as I can to a real life scenario. This time we are staying on traditional Dell rack mounts for both configurations.

This scenario actually occurred prior to the first in the manner of timing. I initially spoke to the customer prior to VSAN 6.2 being release and support for the data reduction services mentioned earlier. In our sizing of the environment for performance, going All Flash VSAN really wasn’t needed so we stayed on the Hybrid path. Configurations where built and quotes where provided.

Fast forward four to five months and we revisit the opportunity as other priorities but the project on the back burner. This time around start with the Hybrid configuration but review what is now available with VSAN 6.2 and data reduction features. This time the customer and I agree to use a 2x data reduction to be even more conservative in our sizing calculations, but still assume the use of erasure coding in the design. Our end goal is to provide 30TB of usable capacity after accounting for a maximum of 70% utilization.

Attribute Hybrid Configuration All Flash Configuration
Server Vendor Dell Dell
Server Model R730XD R730XD
CPU 2 x Intel E5-2660v3 2 x Intel E5-2660v3
Memory 8 x 32GB RDIMMS 8 x 32GB RDIMMS
Network 2 x Intel X520 Dual Port 10GB 2 x Intel X520 Dual Port 10GB
Caching Tier 3 x 400GB Mix Use SSD 2 x 400GB Write Intensive SSD
Capacity Tier 21 x 1TB 7.2K NL-SAS Drives 14 x 480GB Read Intensive SSD
RAID Controller PERC H730 PERC H730
VSAN Licensing 8 x VMware VSAN 6 Standard 10 x VMware VSAN 6 Advance


Again, these are list prices taken from both Dell’s and VMware’s websites.

Attribute Hybrid Configuration All Flash Configuration
List Price for Servers $132,856.00 $129,085.00
List Price for VSAN Licensing $19,960.00

(8 x $2,495.00)


(10 x $3,995.00)

Total Solution List Price $152,816.00 $169,035.00


Hmm. This didn’t quite work out the way I has was thinking it would if you just look at the pricing. If so maybe you wouldn’t justify the additional cost of ~$16,000.00 to go down the All Flash route. Couple things to point out. For the Hybrid configuration we went with a four node cluster, partially to do with the overall capacity we are trying to hit. I just needed a lot of drives. The second reason is my “minimum requirement +1” policy for VSAN. While three nodes will work, I deploy four nodes when I can. Now, if you look at the All Flash configuration, we went with five nodes. This wasn’t needed for the drive slots like in the Hybrid configuration, if designing around the ability to use erasure coding the minimum node count is four, so we deploy with five nodes if possible. So we do have some “future proofing” from a CPU/memory standpoint in the All Flash configuration.

Next up is the actual requirement for hitting that 30TB usable space number. On the Hybrid side we should come in somewhere around the 42TB mark. Calculating the best practice of leaving 30% (per VMware) free on the Datastore, that leaves us just shy of 30TB consumable. For the AF configuration using the conservative 2x data reduction we hit roughly 50TB of usable space and about 35TB of consumable space.

But let’s play with the numbers a little bit. While judging data reduction techniques for any storage device is more art than science, what happens if we up the ante a bit? Take a look at the below:

Data Reduction Total Usable Total Consumable
3 75TB 52TB
4 100TB 70TB
5 126TB 88TB
6 151TB 105TB


Pretty impressive, I would feel safe to say must folks are going to fall somewhere in that range. The roughly 16K price increase in this example might be worth the role of the dice to hit those capacity numbers at all flash speeds.

Wrapping It Up

OK, depending on the scenario/use case going all flash for your VSAN deployment is not necessarily a slam dunk if approaching from only total CAPEX perspective, other factors do apply. But a looking into the second scenario, there is room to grow the storage at both capacity and performance if needed without having to purchase an additional node, thus saving on hardware and licensing costs. In the first scenario, it is obvious that various hardware platforms can come into play to lower the over hardware cost and make going all flash more appealing.

But what I think is safe to say is that looking into an all flash configuration should be included in the conversation. With the advanced feature sets and the price of flash dropping and sizing increasing at what seems to be a weekly pace, you can somewhat “future proof” the investment.

Thanks for reading!