Multi-NVMe (m.2, u.2) adapters that do not require bifurcation

[FutureCow]

Did anyone measured power consumption with these cards?
I have the Quad M.2 PLX 8748 card (M.2 Sleutel Ssd Exp Kaart ANM24PE16 Quad Port PCIe3.0 X16 Met PLX8748 Controller|Geheugenkaart| - AliExpress) With 4 M.2 SSD's it's using around 15-20 Watt's. I think this is quite high at idle (SSD's don't have partitions yet). The SSD's at idle are using 0,5W or even less.

 

[caplam]

I'm about buying one of these cards.
I'm looking for advice on which model to buy.
The host is a hp Z620 (so pcie is gen3). The drives will be hynix PE4010 ( about 2,5GB/s read and 950MB/s write sequential)
I found 2 cards which can suit my needs
both have 4 m2 ports and are gen3
one is a x16 and the other a x8
x16 use a PX8747 chip
x8 use a PLX8724 chip
Considering the ssd will the x8 card be a bottleneck ?
I haven't understood if such cards simply switch available lanes to available ports or if bandwith is distributed on demand?
for example for x8 card do the 4 m2 ports function with 2 lanes all the time?
I prefer buying the x8 card as it's cheaper and will be easier to put in a future server.

 

[TRACKER]

I have the ANM24PE16 model. It works great (i use it for more than a year), but it has small fan, which became noisy after running couple of months. I guess it needs bit of cleaning
Regarding your question, the card "simply switch available lanes to available ports". It does not distribute bandwidth on demand.
Example: if you have two m.2 drives, card utilizes 8 (2x4) of total 16 PCIe lanes available. If you have 4 drives - it utilizes all 4x4 lanes.
The host computer always sees the card utilizing the amount of lanes, available to the PCIe slot, in which card is installed.
E.g. If you install the card in PCIe 2.0 x8 - card will work with total 2.0 x8 bandwidth available. However, each of m.2 drives will be connected to bridge chip at their max speed (3.0 x4).

 

[RolloZ170]

Considering the ssd will the x8 card be a bottleneck ?

only if you read/write to all four SSD same time with full load.

 

[caplam]

Thank you for your answers.
These cards are a nice piece of hardware for old servers.
Pcie3 x8 bandwith is 7876GB/s. I think i'll go with x8 card that should be plenty enough for my unraid pool.

 

[kolokithaki]

hi all, and super thanks for this amazing find! (hopefully amazing for my build/question too!! )

After reading the replies Im thinking of getting a x16 8-port (PLX PEX 8749) as I have 6-7 Micron 7400 U.2 drives spares from another server build, and I want to build an secondary truenas box.

This hardware isnt server grade. I have a couple of ASUS MBs to work with. One X470-F Gaming and one PRIME Z590M PLUS mATX which I currently use with a 9300-8i HBA and SATA SSDs.

Do you foresee any obvious problems that I missed? Most of you in the thread have at least "high PCIe lane" CPUs... I have an 11th gen and a 10th gen intel for this build.
I dont mind "x8" for the full card to be honest.

Any alternative I should check?
Thanks!

 

[mattventura]

Question about these - do these generic switch cards tend to have the sideband stuff for driving backplane LEDs? Or does that tend to require too much platform-specific stuff?

 

[Docop]

On the PEX 8749 card, is any had found a way to mix / readjust the config firmware on those card to have a mix of speed as : getting x8 speed on 2 or 4 port and the rest other 4port run a 4x ? I did see somewhere those card as all using the pex 8749 chip can output all 8 port as 4 group of 2 at 8x speed. But what setting to adjust ... another question..

 

[maya01]

hi, if i were to get a mini pc like one of these https://www.aliexpress.us/item/3256805041034578.html and use a m.2 to pcie (or thunderbolt pcie dock), can i then connect a plx card to it and split it to four pcie using this? https://www.aliexpress.us/item/2255799843496981.html and this for connecting all the cards https://www.aliexpress.us/item/3256801779463200.html . i want to have things like sfp+ cards, a raid card, ethernet card etc instead of nvme/u.2 drives. are there cheaper switch cards which will work just fine?

 

[mattventura]

I'm curious to know if anyone has experience with both Linkreal and Ceacent cards. Linkreal cards seem to cost more than double for the same capabilities. What about them is better?

 

[TRACKER]

I have two Ceacent cards - one with 4xNVMe and PLX chip and one with 2xNVMe without PLX, both work great

 

[EniGmA]

Are there any PCIE 4.0 to M.2 NVME 1.4 cards yet with switch chips? I did some searches on Ali for products and didnt see any 4.0 versions, and looked around on Google and the chips themselves seem to exist but cant find anyone using them for these types of cards

https://www.microchip.com/en-us/products/interface-and-connectivity/pcie-switches

https://www.microchip.com/en-us/product/PM42028

PEX88024

The Broadcom PEX88024 switch allows data center and cloud providers to build flexible, scalable rack infrastructures.

www.broadcom.com

edit: I did actually see a pair of HighPoint models but they are very expensive:

Rocket 1504 NVMe AIC |HighPoint Technologies, Inc.

Rocket 1504 is a 4-port M.2 NVMe AIC Controller, no bifurcation, standalone solution, with dedicated PCIe 4.0 x16, and native in-box NVMe driver support.

www.highpoint-tech.com

I was hoping to find one on AliExpress like the models listed in this thread that was half the price of HighPoint. lol

 

[mattventura]

Are there any PCIE 4.0 to M.2 NVME 1.4 cards yet with switch chips? I did some searches on Ali for products and didnt see any 4.0 versions, and looked around on Google and the chips themselves seem to exist but cant find anyone using them for these types of cards

https://www.microchip.com/en-us/products/interface-and-connectivity/pcie-switches

https://www.microchip.com/en-us/product/PM42028

PEX88024

The Broadcom PEX88024 switch allows data center and cloud providers to build flexible, scalable rack infrastructures.

www.broadcom.com

HighPoint has some, haven't used any of their products myself.

 

[mrpasc]

If you wanna go crazy have a look at this beast:
21(!!) NVME on a PCIE4 x16 FHFL card

Apex Storage X21 Review: The Roaring Twenty-Ones

The Apex Storage X21 offers up 21 Gen4 NVMe M.2 SSDs on a double-width PCIe add-in card (FHFL) with exceptional speed and latency.

www.storagereview.com

 

[hyltcasper]

If you wanna go crazy have a look at this beast:
21(!!) NVME on a PCIE4 x16 FHFL card

Apex Storage X21 Review: The Roaring Twenty-Ones

The Apex Storage X21 offers up 21 Gen4 NVMe M.2 SSDs on a double-width PCIe add-in card (FHFL) with exceptional speed and latency.

www.storagereview.com

It is too expensive. I am going with 2 of Asus 4x cards. So 8 nvme. I hope they will work on dell R730. I will try at the weekend. Now I need storage software that exposes nvme drives as NVME-oF.

 

[mrpasc]

Window Server: Starwind vSAN
For Linux: should be available with all kernel newer than 4.8
SuSe NVME-oF setting up target and initiator

 

[Docop]

i try to get the full manual of the chip : pex 8749 , but i never get reply from broadcom or find it. It should be a pdf (data book) of about 200-300 pages. Any do have it ?

 

[Jamy]

Any reason to buy a card with 8747 over 8724? I'm planning to use 4x NVMe. My motherboard has 16x slots, I'm just wondering if it matters performance wise?

 

[Docop]

same thing. just advances feature that are only used if you do develop your own motherboard for server application. all card used for nvme are like only using about 20% of the feature of the chip. Hightpoint are nice one. The the one with internal and external port do offer upgrade for future as you don't pay double for 2 card, as the first one is too limited.

 

[moschtkopf123@googlemail.]

Hi, attaching myself to this thread to hear your opinions:

I am planning to build a powerful ITX form factor NAS system with NVME storage. I know, this does not sound the best idea, but I especially want to know:
Can I bifurcate the x16 PCIe Slot of the board (I know this needs to be supported in the BIOS) and then pass one x8 part to the GPU and a second one to one of these:

x16 8-port (PLX PEX 8749)
x16 8-port (PLX PEX 8748)

So basically the question is - if I plug the x16 pcie switch card to an x8 slot - I assume only half of the drives will be recognized?