Looking for an M.2 carrier card with PLX (4x M.2 on x16, no RAID)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Toddh

Member
Jan 30, 2013
122
10
18
Well I ordered one. It's direct ship from Supermicro so it will take a bit. I will post once I test it.
 

jzeus

New Member
Jan 22, 2017
19
4
3

Joel

Active Member
Jan 30, 2015
850
191
43
42
So I've been looking at these as well. Does my X9SRL support bifurcation? I suspect not...

x9srl-1.png x9srl-2.png
 
  • Like
Reactions: i386

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Looking at either the AOC-SHG3-4M2P or the Asus Hyper card to put 4 Samsung 950 Pro NVMe's (2.5GB/s read, 1.5GB/s write) in. Will be putting them into a software RAID10. Will the SM card limit me? The Asus card is cheaper so I'll go for that one if the performance is equal or better. My X10 board has full bifurcation support.
 

i386

Well-Known Member
Mar 18, 2016
4,217
1,540
113
34
Germany
You might hit the bottleneck of the x8 interface for sequential reading workloads (4x 2.5gbyte/s > 7.8gbyte/s)
 

i386

Well-Known Member
Mar 18, 2016
4,217
1,540
113
34
Germany
Are you sure? I thought all the raid calculation/verification for data integrity happens on the host cpu and would require data from all devices in the logical volume.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Are you sure? I thought all the raid calculation/verification for data integrity happens on the host cpu and would require data from all devices in the logical volume.
I am indeed NOT sure. I was not aware that was the case.

I'm still torn because I kind of want to keep my x16 slot available in case I need/want to add a GPU in the future. I doubt my workloads are going to even hit the 7.8GB/s limit of the x8 card.
 
Last edited:

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
I read somewhere once aeons ago that btrfs didn't stripe reads from a RAID1 or RAID10 device - I don't know if that's still the case (I would hope not) but last I looked the 4-drive RAID10 had the same read speed as a 2-drive RAID0. I've only used it myself inside VMs (I take grave exception to what I see as it's stupid refusal to mount an array with a broken or missing drive without manually specifying the "degraded" mount option).

As a softraid the data does have to go through CPU/memory, thus data from all four drives will need to go over PCIe.

In any case, I think too much truck is put in "...but you'll bottleneck on sequential workloads" since in my experience almost no workload is sequential (and it's a rare workload that needs sustained >8GB/s) but as per my caveat I don't think btrfs will reach the theoretical limits.
 
Last edited:

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
I read somewhere once aeons ago that btrfs didn't stripe reads from a RAID1 or RAID10 device - I don't know if that's still the case (I would hope not) but last I looked the 4-drive RAID10 had the same read speed as a 2-drive RAID0.
Did you mean the same as a two drive RAID1?
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Nope, two-drive RAID0. Like I said my info is probably out of date but when I last checked btrfs didn't seem to stripe reads very well, if at all.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Nope, two-drive RAID0. Like I said my info is probably out of date but when I last checked btrfs didn't seem to stripe reads very well, if at all.
Its probably me who's info is wrong. I didn't realize RAID10 reads were off all 4 disks. I though it just read off each striped mirror but not all 4 drives.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
It depends on the implementation, you could quite easily have a RAID10 that only ever read off one set of drives if you wanted to (although that'd be rather stupid). But the thrust of things was that btrfs certainly didn't use this particular optimisation - not that I think it'll make a significant difference for your workload.

Regardless of the optimisation though, writes would always need to go to all discs and reads can conceivably come from all discs so ideally your PCIe bandwidth would be able to sustain maximum seq read of all devices at once. With btrfs this probably won't be an issue but given your PCIe constraints it's not one I'd worry about anyway.
 
Last edited:

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
It depends on the implementation, you could quite easily have a RAID10 that only ever read off one drive if you wanted to (although that'd be rather stupid). But the thrust of things was that btrfs certainly didn't use this particular optimisation - not that I think it'll make a significant difference for your workload.

Regardless of the optimisation though, writes would always need to go to all discs and reads can conceivably come from all discs so ideally your PCIe bandwidth would be able to sustain maximum seq read of all devices at once. With btrfs this probably won't be an issue but given your PCIe constraints it's not one I'd worry about anyway.
I agree, workload wise I'm not worried about the PCIe x8 limiting me. I wish there was another x8 option other than the SM since I don't need the PLX chip.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Man, trying to source a quad M.2 card right now is tough. Everything is showing as either out of stock, discontinued, or $200+.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Asus seemed to give up on their M2 cards pretty early. ASRock did their own take on this called the Ultra Quad and that's still available in the UK at least, although at prices that make it cost the same as the Supermicro AOC-SHG3-4M2P-O with PLX chip (£140 on amazon UK for the Ultra Quad vs. £135 for the AOC-SHG3-4M2P at LambdaTek).

Somewhat surprised they aren't more common stateside...
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Asus seemed to give up on their M2 cards pretty early. ASRock did their own take on this called the Ultra Quad and that's still available in the UK at least, although at prices that make it cost the same as the Supermicro AOC-SHG3-4M2P-O with PLX chip (£140 on amazon UK for the Ultra Quad vs. £135 for the AOC-SHG3-4M2P at LambdaTek).

Somewhat surprised they aren't more common stateside...

Yea that's what I'm seeing. Going to try and source a used one for a few days. Otherwise I'll have to shell out for the SM PLX card.

EDIT: I guess this is a pretty decent price for the ASRock card.

NeweggBusiness - ASRock ULTRA QUAD M.2 CARD PCI-e to 4x M.2 Card