Performance advice (3x Slow-drives - RAID5) on LSI RAID for ESXI

DanAnd

Member
Jul 25, 2016
58
47
18
46
Dortmund, Germany
www.danand.de
Hi,

I have a small ESXI host in my homelab which runs a few hours a day for experimental work.
The parts of the machine are chosen to be either low cost or low power usage:

2x E52650L on an ZTSYSTEM ACADIA12 board
128 GB DDR3 ECC
1x SSD Boot drive connected to the onboard Sata-Port
3x 3TB Sata drives connect to a LSI SAS2108 Controller (512MB Cache + BBU) -> Fujitsu D2616

In the first days I ran these three 3TB drives directly on the Mainboard with no RAID at all, but since I would like to have a bit of fault tolerance I added the D2616 (SAS2108) RAID controller and build a RAID 5 out of these 3 drives.

What I encountered is that the machine is feels slower than using a single drive before.

The maximum throughput I can see is around :

fio 50% read/ 50% write:
512b Blocks: 0,3-0,8 MB/s and 640-1740 IO/s
1024b Blocks: 0,7-1,6 MB/s and 745-1580 IO/s
4096b Blocks: 0,6-12,6 MB/s and 150-3130 IO/s
512kb Blocks: 70-133 MB/s and 140-255 IO/s
1024kb Blocks: 80-180 MB/s and 75-175 IO/s
4096kb Blocks: 75-130 MB/s and 20-30 IO/s​

storcli:

VD LIST :
=======
-------------------------------------------------------------
DG/VD TYPE State Access Consist Cache Cac sCC Size Name
-------------------------------------------------------------
0/0 RAID5 Optl RW Yes RWBC - OFF 5.457 TB
-------------------------------------------------------------

PD LIST :
=======
--------------------------------------------------------------------------------
EID:Slt DID State DG Size Intf Med SED PI SeSz Model Sp Type
--------------------------------------------------------------------------------
252:0 3 Onln 0 2.728 TB SATA HDD N N 512B WDC WD30EZRX-00MMMB0 U -
252:1 2 Onln 0 2.728 TB SATA HDD N N 512B WDC WD30EZRX-00MMMB0 U -
252:2 1 Onln 0 2.728 TB SATA HDD N N 512B WDC WD30EZRX-32D8PB0 U -
--------------------------------------------------------------------------------

BBU_Info :
========
-----------------------------------------------------------------------
Model State RetentionTime Temp Mode MfgDate Next Learn
-----------------------------------------------------------------------
iBBU08 Optimal 48 hours + 45C 5 2014/05/27 2018/06/19 01:48:31
-----------------------------------------------------------------------
Mode 5: 48+ Hrs retention with a non-transparent learn cycle
and moderate service life.​




The more subjective feeling is that the VCenter Appliance runs extremely slow. Also Windows Installations got a pretty slow-down from switching from a single Harddisk to the RAID Controller.


I have two possibilities to advance my server with no or very low cost:

1) Exchange the D2616 (SAS2108 with 512MB Cache+BBU) for a IBM M5110 (SAS 2208 with 1GB Cache+BBU)
2) Running the RAID5 with a 4th drive


Surely the SAS2208 does have PCIe 3.0 x8 , but the C602 chipset of my mainboard is still PCIe 2.0. Will a SAS2208 Controller enhance the RAID5 performance im comparison to the SAS2108 ?

What else to consider ?

Thanks alot.

Daniel
 
Last edited:

i386

Well-Known Member
Mar 18, 2016
2,563
724
113
32
Germany
With raid 5/6 you have usually the write performance of a single drive, but reads of n-1 for raid 5, n-2 for raid 6.
I think for the 50%r / 50%w workload these are good numbers.

What else to consider ?
The firmware.
If it's not the "original" lsi/broadcom firmware it's possible that the firmware was modifid for that oem (eg lower max queue depth).