VMware vSAN Performance :-(

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Yves

Member
Apr 4, 2017
65
15
8
38
ECS is not scaleIO though... atleast not as I know but would need to read up on that.
I am not sure... I just rememberd that I read something about ScaleIO and ECS... But not really sure.


If I am not to tired later I will run tests on Starwinds vSAN on my Nodes. And tomorrow my new NICs will arrive so I will do more testing on the weekend


Sent from my mobile
 
Last edited:

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
I am about to next week setup a 3/4 node vSAN with just a single disk group each is probably 1 cache and 2 spinners just for my guys to get some experience doing stuff they can’t do on the productive clusters, I would have enough SAS SSD to do all flash but no NVMe, since it’s all HPE and and Cisco ACI it is a known good config of ours in terms of good performance. So may run a few tests and see how it performs as well although the idea was just to use spares to make a lab environment.
 

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
That definitely will be interesting.
At least with a support contract and officially sanctioned hardware (?) you can bug vmware about the horrible speed;)
 

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
Hm similar results, no help. Disappointing but not unexpected.
As I said, its not designed for few users.
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
If I am not to tired later I will run tests on Starwinds vSAN on my Nodes. And tomorrow my new NICs will arrive so I will do more testing on the weekend
Just found that old thread again here https://forums.servethehome.com/ind...vers-build-starwind-vsan-or-vmware-vsan.8652/

Seems that Starwind and 4 Node S2D might be an option that is not inhibited by extreme performance loss... so interested in your Starwind results quite a bit :)

Has anyone experience with only using 1 Cache drive on S2D? Found a thread where they said it would be ok but the official guide still says differently - going with a single cache drive on 3+ nodes would be a lot simpler/cheaper ...
 
Last edited:

ecosse

Active Member
Jul 2, 2013
463
111
43
On the Starwind side I'm as interested in how it can be managed once the GUI is not available (license expired). I read one thread on the starwind forum (which I find unintelligible for the most part) that GUI-less was a nightmare)
 

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
So just tried to setup a Starwind vsan myself with nvme drives ... do i see that right that I can't actually pass through a device but have to create a virtual disk on a drive and supply that as harddrive to the vm?
 

Yves

Member
Apr 4, 2017
65
15
8
38
So just tried to setup a Starwind vsan myself with nvme drives ... do i see that right that I can't actually pass through a device but have to create a virtual disk on a drive and supply that as harddrive to the vm?
Hehe, I am just doing the same right now. Yes, as far as I see u put it on top of the filesystem...

Had a conf call yesturday with a Starwind Engineer. We ran into some 10Gbps network issues... That vmxnet3 on Win2k16 does not really achive 10Gbps... On CentOS it run with 24 Gbps... tested with iperf3


Sent from my mobile
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
Starwind likes you raid controller etc, it’s not after the raw storage like the other products out there, that kind of useful when your dealing with say the regular HPE, Lenovo, Dell server that has a raid controller in place out of the factory and it makes it easy for those installs which I assume are the target market really.
 

Yves

Member
Apr 4, 2017
65
15
8
38
Quick update... Sorry for the delay... we have very very good weather this weekend for the first time this year... so I had not as much time to nerd around as I thought I will have.

Here is my first test of the Starwind vSAN after changing the NICs in my SDS Cluster (I will update some pics in the other thread soonish).

This is 1x 10GBit Link directly between native Test Client (Win2k16 DC) and native Win 2k16 DC with Starwind vSAN on top (Jumbo 9k + TCP Chimney Offload disabled)


This is 1x 10GBit Link directly between native Test Client (Win2k16 DC) and native Win 2k16 DC with Starwind vSAN on top (No Jumbo Frames + TCP Chimney Offload enabled)
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
Hm looks promising... I had some network issues too that I need to fix first.
So you have a physical installation with Raid beneath it now? How do you handle the Optane then?
 

Yves

Member
Apr 4, 2017
65
15
8
38
Hm looks promising... I had some network issues too that I need to fix first.
So you have a physical installation with Raid beneath it now? How do you handle the Optane then?
I installed Win2k16 on a RAID1 I made between the 2x Intel DC S4600. So this is my C drive and the Intel NVME is D drive which I use as decoration... to be honest... since the max link speed I will ever achive... is anyway 20GBit limited to the mezzanine cards inside the compute cluster nodes... I think I can resell the brand new 900p and buy more S4600 drives...
 

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
Could you do me a favour and run with QD=1 ?
I am looking for single user max perf and that would be an interesting data point :)
 

Yves

Member
Apr 4, 2017
65
15
8
38
Could you do me a favour and run with QD=1 ?
I am looking for single user max perf and that would be an interesting data point :)
Unfortunately ATTO does only have Queue Depth 2 as the lowest value... But I can run any benchmark you want me to... Just tell me what you want me to run ;-)

This is with a queue depth of 2:
 
Last edited:

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
Well if you can run Crystal Disk mark (v5) with 1/8/16 GB and potentially fio with sth like the attached script (shell based so you have to adjust)
that would be great; but if not no problem either:)
Code:
#!/bin/bash   

env="compuverde_2_nvme_2"
echo "4kreadtest_d1_j1 " >$env
for RUN in $(seq 1 5)
do
./fio --filename=test --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=100 --iodepth=1 --numjobs=1 --runtime=60 --group_reporting --name=4kreadtest  >>${env}
sleep 20
done
echo "4kwritetest_d1_j1 " >>$env
for RUN in $(seq 1 5)
do
./fio --filename=test --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=0 --iodepth=1 --numjobs=1 --runtime=60 --group_reporting --name=4kwritetest  >>${env}
sleep 20
done
echo "4kreadtest_d16_j16 " >$env
for RUN in $(seq 1 5)
do
./fio --filename=test --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=100 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4kreadtest  >>${env}
sleep 20
done
echo "4kwritetest_d16_j16 " >>$env
for RUN in $(seq 1 5)
do
./fio --filename=test --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=0 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4kwritetest  >>${env}
sleep 20
done
 

Yves

Member
Apr 4, 2017
65
15
8
38
Here are the results:

CrystalMark V5 1GB


CrystalMark V5 8GB


CrystalMark V5 16GB


fio is a bit more complicated since I am running a win2k16 dc client... I have to see if I can make it work