Trying to push 10GB/s storage from a Intel Xeon D-1540

[Patrick]

I want to run a few tests to see how far I can get on the Intel Xeon D-1540 this weekend. Here is my plan:

m.2 slot: Samsung XP941 256GB = 1GB/s

SATA SSDs: likely 4x 800GB Intel S3500 + ? I figure it will not be hard to get to 3GB/s if I can use all 6 drives

PCIe 3.0 x16 slot. I would imagine I will get stuck here. I could use the Virident FlashMAX II 550GB (that was me) which is spec'd for ~1.5GB/s reads (5.5GB/s total). The Intel P3700 can push around 3GB/s which only would get me to 7GB/s total. The other option is I do LSI SAS 3008 to a SAS expander and try filling the ports to see if I can get at least 6GB/s but maybe 8GB/s.

Here is my question: What is going to be the best way to validate this? IOMeter sequential writes using multiple workers/ disk? E.g. run IOMeter in a single run just duplicating for all storage devices on the server? I have never tried that before.

 

[Patriot]

Are you talking Gb GB or GiB?
10Gbps being the nic... that is easy to do.
GiB base 2 GB base10 not so much.

with 16 Hitachi SAS3 drives I was pushing 1M iops at 4k but under 7GB/s at 2-4MB
You would need 2 cards atleast...

 

[markpower28]

If you plan to test from a Windows based OS. I would create a disk for each type of SSD (maybe raid 0). Then for each disk using iometer create 4 x worker group with each worker on 1 MB read or write. In your case for 4 different type of SSD will be 4 drive letters. And 4 x worker group for each drive that will be 16 worker group in the test.

It will be very interesting to see the result. Make sure do a random IO testing as well. I do agree with Patriot, the on board NIC only able to push up to 2 GB/s, what you plan to do with the rest 8 GB/S?

 

[Entz]

Does the PCI-e 16x slot support bifurcation? That would let you split it out and connect 2 LSI3008s (assuming your current cards are the standard 8x variety)

 

[Patrick]

Does the PCI-e 16x slot support bifurcation? That would let you split it out and connect 2 LSI3008s (assuming your current cards are the standard 8x variety)

I am not sure on the implementation of this x16. In general all PCIe 3.0 controllers on Xeon D can go down to x4 and the PCIe 2.0 can go down to x1.

@Patriot I think this is just local IOPS. So the 1M IOMeter testing may make sense.

@markpower28 that is what I was thinking.

This is not a test in practicality, rather one in seeing what I can get.

 

[markpower28]

The HBA is suppose to hand 12 GB/s. Let's see what will take to max it out !

 

[dba]

I want to run a few tests to see how far I can get on the Intel Xeon D-1540 this weekend. Here is my plan:

m.2 slot: Samsung XP941 256GB = 1GB/s

SATA SSDs: likely 4x 800GB Intel S3500 + ? I figure it will not be hard to get to 3GB/s if I can use all 6 drives

PCIe 3.0 x16 slot. I would imagine I will get stuck here. I could use the Virident FlashMAX II 550GB (that was me) which is spec'd for ~1.5GB/s reads (5.5GB/s total). The Intel P3700 can push around 3GB/s which only would get me to 7GB/s total. The other option is I do LSI SAS 3008 to a SAS expander and try filling the ports to see if I can get at least 6GB/s but maybe 8GB/s.

Here is my question: What is going to be the best way to validate this? IOMeter sequential writes using multiple workers/ disk? E.g. run IOMeter in a single run just duplicating for all storage devices on the server? I have never tried that before.

Set up one unformatted disk per SSD device. Use IOMeter of course. In IOMeter, put a check next to each disk so that you are testing across all disks. Small test files (say 1GB) will maximize IOPS since you are testing the cache more than the drive but large (tens to hundreds of GB) test files will maximize throughput. Start with one worker per CPU core. 1MB or 2MB sequential reads or writes will probably give you the highest throughput. Vary the queue depth to find the highest throughput - your "vanity" result. If you have time, vary the number of workers plus or minus to see if you can beat that number. If you are on Windows, you can get very good results with an out-of-the-box OS config. On Linux, you'll probably see disappointing IOMeter results unless you start tuning IO, so I'd stick with Windows for this type of quick test.

 

[Patrick]

This one got put on hold a bit. Ended up using a Sans Digital MobileSTOR MS28X 8-Bay 2.5 which worked well when I got SFF-8088 <-> SFF-8644 cable. I realized I needed to get content for this week out and have limited time. Did get 7x SSD quick benchmarks all queued up in the system though.