Dell P31H2: $99

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

UhClem

just another Bozo on the bus
Jun 26, 2012
436
250
63
NH, USA
Those all seem out of stock, but there's this link with 9 available right now:

And all 9 of these sold, at asking price of $120, to one buyer ... must be a miner (or a flipper to miners).

The used market for all things storage has gone crazy. Even Sas2 expanders: few months ago, numerous ebay sellers of the Lenovo 03x3834 for US$13-18. Now $90+.
 

Fritz

Well-Known Member
Apr 6, 2015
3,378
1,384
113
70
A few months back I scored a Supermicro sas2 846EL backplane on eBay for $89. Tempted to put it back on eBay.
 

Jaket

Active Member
Jan 4, 2017
238
126
43
Seattle, New York
purevoltage.com
And all 9 of these sold, at asking price of $120, to one buyer ... must be a miner (or a flipper to miners).

The used market for all things storage has gone crazy. Even Sas2 expanders: few months ago, numerous ebay sellers of the Lenovo 03x3834 for US$13-18. Now $90+.
These are quite useable for anyone wanting nvme storage. I picked up two. I know someone else who bought 12+
They are great for the data center and allowing 8+ nvme's into some customized solutions that otherwise would require a lot more costly chassis. We bought ours so that we could offer more nvme drives to specific customers in other systems. Saves us from having to buy some custom sytems that the customers really don't require other than massive nvme storage. I'm sure some people are using for Chia.
 

klui

Well-Known Member
Feb 3, 2019
832
455
63
Did anyone get theirs working? The only thing I can see are 5 PLX entries through lspci. lspci states it's an 8-port switch. I thought there's only 4 ports?

I used a Supermicro CBL-SAST-957 (MiniSAS HD SFF-8643 to U.2 PCIe SFF-8639 w/ power cable but the drive (Intel and Samsung) don't show up. There's no /dev/nvme* under Ubuntu 20.04 LTS.
 

azsrz

New Member
Apr 30, 2021
26
8
3
Did anyone get theirs working? The only thing I can see are 5 PLX entries through lspci. lspci states it's an 8-port switch. I thought there's only 4 ports?

I used a Supermicro CBL-SAST-957 (MiniSAS HD SFF-8643 to U.2 PCIe SFF-8639 w/ power cable but the drive (Intel and Samsung) don't show up. There's no /dev/nvme* under Ubuntu 20.04 LTS.
same here. nvme doesn't show up either connecting to this directly or to a backplane. Not sure what went wrong since most people here seem fine with it.
 

klui

Well-Known Member
Feb 3, 2019
832
455
63
I'd like to know a working configuration: either proper cable or backplane or else I'll just request a return.
 

UhClem

just another Bozo on the bus
Jun 26, 2012
436
250
63
NH, USA
And all 9 of these sold, at asking price of $120, to one buyer ... must be a miner (or a flipper to miners).
... They are great for the data center and allowing 8+ nvme's into some customized solutions that otherwise would require a lot more costly chassis. We bought ours so that we could offer more nvme drives to specific customers in other systems. Saves us from having to buy some custom sytems that the customers really don't require other than massive nvme storage....
... great ...?? How about "pretty good" ? :) Aren't (most of) your host systems capable of quadfurcating (at least some/most of) their x16 slots? If so, the main feature of this card (the PCIe switch) is being wasted. You'd be just as well-served with a switch-less card for less than half the cost, and none of the heat/power usage.quadf.jpg

... massive nvme storage ... Over-subscription is your friend here. 4:1 cards like Intel's (can't find model#s, sorry) that are PCIe3x8-to-8xNVMe, or 2:1 cards that do PCIe3x16-to-8xNVMe. In this scenario, PCIe switches really shine--not just in allowing more connections, but also potentially doubling your effective PCIe bandwidth. (Someone at your company better know what I mean there [or you're at a serious competitive disadvantage:(].)

... flipper ...Wait a minute--should I be helping you?--you're a flipper too! But you do add value, etc. :):)
 

UhClem

just another Bozo on the bus
Jun 26, 2012
436
250
63
NH, USA
Did anyone get theirs working? The only thing I can see are 5 PLX entries through lspci. lspci states it's an 8-port switch. I thought there's only 4 ports?

I used a Supermicro CBL-SAST-957 (MiniSAS HD SFF-8643 to U.2 PCIe SFF-8639 w/ power cable but the drive (Intel and Samsung) don't show up. There's no /dev/nvme* under Ubuntu 20.04 LTS.
The card contains a PCIe switch chip (a PLX PEX8734). That chip can be configured with up to 8 ports, and up to 32 lanes. On this card, the chip is configured with 5 ports (1 upstream @x16, and 4 downstream @x4 each) [these are the 5 reported by lspci], and 32 lanes [16 + (4*4)].

(I don't have this card, but) That should be the correct cable. On a different, but similar, card, I use various generics (Wordima & CableCreation) of SFF8643-to-SFF8639. If you only bought/have one cable, maybe it's a dud.
 
  • Like
Reactions: 2bluesc and klui

klui

Well-Known Member
Feb 3, 2019
832
455
63
The card contains a PCIe switch chip (a PLX PEX8734). That chip can be configured with up to 8 ports, and up to 32 lanes. On this card, the chip is configured with 5 ports (1 upstream @x16, and 4 downstream @x4 each) [these are the 5 reported by lspci], and 32 lanes [16 + (4*4)].

(I don't have this card, but) That should be the correct cable. On a different, but similar, card, I use various generics (Wordima & CableCreation) of SFF8643-to-SFF8639. If you only bought/have one cable, maybe it's a dud.
I got 2 cables from Supermicro's eStore. They were the least expensive than what you can buy from eBay. Both cables had the same behavior: no drive seen. Used multiple motherboards and slots were not bifurcated. I was hoping this cheaper card would give me NVMe capability instead of something from Supermicro. The only issue is there is no documentation for this card and there is a set of jumpers on the bottom of the card. Some eBay listing show half of them bridged, but this one doesn't have any installed.
 

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com
Did anyone get theirs working? The only thing I can see are 5 PLX entries through lspci. lspci states it's an 8-port switch. I thought there's only 4 ports?

I used a Supermicro CBL-SAST-957 (MiniSAS HD SFF-8643 to U.2 PCIe SFF-8639 w/ power cable but the drive (Intel and Samsung) don't show up. There's no /dev/nvme* under Ubuntu 20.04 LTS.
I assume for the 8 ports -- it's something like 4 to the drives, 4 to the pcie bus?
 
  • Like
Reactions: klui

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com

UhClem

just another Bozo on the bus
Jun 26, 2012
436
250
63
NH, USA
[Edit: connect one, or both, of your cables/drives; then a fresh boot; then ...]
@klui , Pls post the output from:
Code:
lspci -vv -d 10b5:: | grep -e LnkCap -e LnkSta:
 
Last edited:

klui

Well-Known Member
Feb 3, 2019
832
455
63
[Edit: connect one, or both, of your cables/drives; then a fresh boot; then ...]
@klui , Pls post the output from:
Code:
lspci -vv -d 10b5:: | grep -e LnkCap -e LnkSta:
Here is the output from Proxmox:
Code:
root@pve:~# lspci -vv -d 10b5::|grep -e LnkCap -e LnkSta:
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 8GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #4, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #6, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #7, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
If I use the 8643 miniSAS HD to 8639 (U.2) cable I get just one pciehp: Failed to check link status entry in syslog. If I connect it to a backplane I got continual entries. The port does attempt to try to link because when I connect the cables to different headers, the message identifies the correct PCIe port.
Code:
[ 2.833285] pcieport 0000:d9:04.0: pciehp: Slot #4 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 2.833624] pcieport 0000:d9:05.0: pciehp: Slot #5 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 2.833693] pcieport 0000:d9:05.0: pciehp: Slot(5): Card present
[ 2.833948] pcieport 0000:d9:06.0: pciehp: Slot #6 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 2.834246] pcieport 0000:d9:07.0: pciehp: Slot #7 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 4.857942] pcieport 0000:d9:05.0: pciehp: Failed to check link status
root@pve:~# lspci|grep PLX
d8:00.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:04.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:05.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:06.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:07.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
@funkywizard, does your card have jumpers installed?

EDIT: here are the syslog entries if I disconnect and reconnect to the other port on the card
Code:
Jun  9 15:59:46 pve kernel: [ 1538.487019] pcieport 0000:d9:07.0: pciehp: Slot(7): Card present
Jun 9 15:59:48 pve kernel: [ 1540.511546] pcieport 0000:d9:07.0: Data Link Layer Link Active not set in 1000 msec
Jun 9 15:59:48 pve kernel: [ 1540.511550] pcieport 0000:d9:07.0: pciehp: Failed to check link status
 
Last edited:

azsrz

New Member
Apr 30, 2021
26
8
3
Here is the output from Proxmox:
Code:
root@pve:~# lspci -vv -d 10b5::|grep -e LnkCap -e LnkSta:
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 8GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #4, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #6, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkCap: Port #7, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <4us, L1 <4us
LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
If I use the 8643 miniSAS HD to 8639 (U.2) cable I get just one pciehp: Failed to check link status entry in syslog. If I connect it to a backplane I got continual entries. The port does attempt to try to link because when I connect the cables to different headers, the message identifies the correct PCIe port.
Code:
[ 2.833285] pcieport 0000:d9:04.0: pciehp: Slot #4 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 2.833624] pcieport 0000:d9:05.0: pciehp: Slot #5 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 2.833693] pcieport 0000:d9:05.0: pciehp: Slot(5): Card present
[ 2.833948] pcieport 0000:d9:06.0: pciehp: Slot #6 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 2.834246] pcieport 0000:d9:07.0: pciehp: Slot #7 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+
[ 4.857942] pcieport 0000:d9:05.0: pciehp: Failed to check link status
root@pve:~# lspci|grep PLX
d8:00.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:04.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:05.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:06.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
d9:07.0 PCI bridge: PLX Technology, Inc. PEX 8734 32-lane, 8-Port PCI Express Gen 3 (8.0GT/s) Switch (rev ab)
@funkywizard, does your card have jumpers installed?

EDIT: here are the syslog entries if I disconnect and reconnect to the other port on the card
Code:
Jun  9 15:59:46 pve kernel: [ 1538.487019] pcieport 0000:d9:07.0: pciehp: Slot(7): Card present
Jun 9 15:59:48 pve kernel: [ 1540.511546] pcieport 0000:d9:07.0: Data Link Layer Link Active not set in 1000 msec
Jun 9 15:59:48 pve kernel: [ 1540.511550] pcieport 0000:d9:07.0: pciehp: Failed to check link status
I got the an infinite loop of the "failed to check link status errors" with one set of the cable I bought from amazon at startup. With two known good cables I got nothing.
 

UhClem

just another Bozo on the bus
Jun 26, 2012
436
250
63
NH, USA
Here is the output from Proxmox:
...
Whoa!! When I wrote:
connect one, or both, of your cables/drives; then a fresh boot; then ... Pls post the output from: lspci ...
the information I had, from you, regarding your operating environment, was from
There's no /dev/nvme* under Ubuntu 20.04 LTS.
So, is it actually the case that you are running Ubuntu under Proxmox?
 

klui

Well-Known Member
Feb 3, 2019
832
455
63
Sorry for the confusion. It happens in both Proxmox and Ubuntu. Proxmox is my other server that has native NVMe and those native ports work.