HighPoint 7540 SSD RAID - slow speed?

[FIGURATIVESPEECH]

I just installed and configured the HighPoint 7540 SSD RAID card in my brand new computer. It's configured in RAID-10 with 8x4TB WD Black SN850X (7300mb/s). I ran Crystalmark test. I'm getting 50% slower than expected performance. I am getting 14,200 MB/s read, 7000 MB/s write. Advertised (and tested by others) speeds in RAID-10 for this card should have been 28,000 MB/s read and 14,000 MB/s write.

My motherboard is ASUS ProArt Z790. CPU is Intel i9-13900KS. I have 7 drives in total:
1. 4 NVME slots on the motherboard are occupied by 4TB WD Black SN850X NVMEs.
2. The 14.5TB drive is the HighPoint 7540 RAID card.
3. Two 16 TB hard drives.

OS is Windows 11 Pro 64 bit. Computer has 192 GB DDR5 RAM.

Does any one know why I'm getting reduced R/W speeds?

 

[CyklonDX]

lets see...

what are the temps when under stress + plx chip temp?
what are the cpu util when writing & reading?

Is it sitting in pcie gen4 x16? I think it may be sitting on gen5 x8 port, because first one is occupied (thus running at gen4 x8 bandwidth)

 

[FIGURATIVESPEECH]

lets see...

what are the temps when under stress + plx chip temp?
what are the cpu util when writing & reading?

Is it sitting in pcie gen4 x16? I think it may be sitting on gen5 x8 port, because first one is occupied (thus running at gen4 x8 bandwidth)

I'll check on the temps and util, but the card is sitting on the bottom slot which is Gen4 x 16. Gen5 slot is occupied by GPU. I'm using all 4 NVME slots on the motherboard. Could that be the cause? Not enough bandwidth?

 

[mrpasc]

If I read and understood the manual correctly:
The PCIE x16 Gen4 slot is connected to the Z790 and shares lanes with M2_2, M2_3 and M2_4 Slots.
So z790 offers 20 PCIE G4 lanes in total, thus 8 lanes remaining for the x16 G4 slot if all three M.2 slots are in use. Would explain your measure.

 

[i386]

WD Black SN850X

I wouldn't expect consumer drives to keep their performance numbers up under sustained load (the raid itself)

 

[FIGURATIVESPEECH]

If I read and understood the manual correctly:
The PCIE x16 Gen4 slot is connected to the Z790 and shares lanes with M2_2, M2_3 and M2_4 Slots.
So z790 offers 20 PCIE G4 lanes in total, thus 8 lanes remaining for the x16 G4 slot if all three M.2 slots are in use. Would explain your measure.
View attachment 29062

Actually, this board has 4 m.2 slots, all of which are used. How many (and which ones) do I need to take out in order to have the RAID card read and write at full 28,000 MB/s?

 

[mrpasc]

The 3 ones connected to Z790. Should be M.2_2,3,4.

 

[FIGURATIVESPEECH]

The 3 ones connected to Z790. Should be M.2_2,3,4.

well, all 4 are connected to Z790 motheboard, including the C: drive with Windows

 

[mrpasc]

Have a look at the schematic I've posted:
Slot M2_1 is connected directly to CPU, as well as the both PCIE G5 slots. So those do not limit the bandwidth of the G4 slots and can stay.
Slot M2_2 M2_3 and M2_4 are connected to Z790 and share lanes with x16 G4. SO remove M2_3 and M2_4 as a first try to get the missing 8 lanes to x16 G4.

 

[FIGURATIVESPEECH]

Have a look at the schematic I've posted:
Slot M2_1 is connected directly to CPU, as well as the both PCIE G5 slots. So those do not limit the bandwidth of the G4 slots and can stay.
Slot M2_2 M2_3 and M2_4 are connected to Z790 and share lanes with x16 G4. SO remove M2_3 and M2_4 as a first try to get the missing 8 lanes to x16 G4.

sorry for a nooby question, i just want to understand this.
The RAID card takes 16 G4 lanes, right? The board allows maximum of 20 G4 lanes. So 16 are gone for the RAID card and the remaining 4 lanes are available for one m.2 NVMe SSD, let's say M2_2. The CPU has 4 available G4 lanes and this is where M2_1 (system drive) is connected. Did I get this right?
So by NOT plugging M2_3 & M2_4, we leave 8 G4 lanes of the motherboard needed by the RAID card so it can run at full 16 lanes. Correct?

 

[mrpasc]

Yepp, thats the way it should work. Might be you need to disable M.2_3 and M.2_4 in Bios and set PCIE G4 x16 explicit to x16 bandwith.

 

[FIGURATIVESPEECH]

Yepp, thats the way it should work. Might be you need to disable M.2_3 and M.2_4 in Bios and set PCIE G4 x16 explicit to x16 bandwith.

Hmmm. I took out the 3rd and 4th M2. RAID showing same speed. I changed the slot of the card from Gen4 to Gen5, still the same result. I'm worried now.

 

[i386]

what strip size (size of a segment per ssd) did you use?

 

[mrpasc]

Well, swapping to Gen5 slot does not help, as this slot is x8 if the other Gen5 is occupied by any graphics card.
Did you double check all BIOS settings regarding those Gen4 slots? Did you try to disable M2_3 and M2_4 in Bios. Did you check settings for Gen4 x16 slot? Any splitting/bifurcation left by ancident?

 

[FIGURATIVESPEECH]

what strip size (size of a segment per ssd) did you use?

Block Size is 512k. Sector Size is 512B

 

[FIGURATIVESPEECH]

Well, swapping to Gen5 slot does not help, as this slot is x8 if the other Gen5 is occupied by any graphics card.
Did you double check all BIOS settings regarding those Gen4 slots? Did you try to disable M2_3 and M2_4 in Bios. Did you check settings for Gen4 x16 slot? Any splitting/bifurcation left by ancident?

Here's the full HWINFO report of my computer, if you can make anything out. There's no bifurcation, i checked. It Gen 4 slot runs at full 16 lanes. It's not possible to disable the M2_3 and 4, but they are taken out for now.
report.HTM

 

[mrpasc]

Well, sorry to say, I do not own such a board nor am I an expert, so out of ideas here...
But the marked figures below make me still thinking that the slot the HP card is connected to does not run at full spec.

 

[FIGURATIVESPEECH]

Well, sorry to say, I do not own such a board nor am I an expert, so out of ideas here...
But the marked figures below make me still thinking that the slot the HP card is connected to does not run at full spec.
View attachment 29081

no, you're on the money there. The slot is definitely the issue, it's using 8 lanes only. We have a bifurcation "problem". I don't know how i can fix this though because BIOS is saying all 16 lanes are enabled on PCI Gen 4. Where else can i look?

 

[FIGURATIVESPEECH]

Well, sorry to say, I do not own such a board nor am I an expert, so out of ideas here...
But the marked figures below make me still thinking that the slot the HP card is connected to does not run at full spec.
View attachment 29081

confused. As per HWINFO report, this motherboard has 1 GEN5 and 2 GEN4 PCIEX16 slots. But my motherboard has 2 GEN5 and 1 GEN4 PCIEX16 slots. What gives?

 

[FIGURATIVESPEECH]

Well, sorry to say, I do not own such a board nor am I an expert, so out of ideas here...
But the marked figures below make me still thinking that the slot the HP card is connected to does not run at full spec.
View attachment 29081

Two GEN5 and one GEN4 slot...