Dell Poweredge T640 8x3.5" chassis AND nvmes

[dmitry.n.medvedev]

good morning, nice community.

The Context: I have purchased a used Dell T640 in the 8x3.5" chassis with a couple of silver CPUs, a bit of ram, and a BOSS card. I would like to convert it to a fast workstation w/ FreeBSD and ZFS. By fast I mean I would like to employ U.2 NVME drives.

The Problem: the chassis I own ( according to Dell documents ) does not support U.2 NVMEs. Some other chassis supports an extension kit with a second backplane which then supports NVMEs. I have talked to communities ( on a couple of platforms ) and the people have separated into two groups. Those who firmly believe that my chassis will never support U.2 NVMEs; and those who aren't so pessimistic ( but haven't tried to accomplish it themselves ).

The Question: do you think that this DELL POWEREDGE T640 8 SCHACHT SSD PCIE HDD NVME EXPANDER RÜCKWANDPLATINE, KÄFIG UND KABEL | eBay could be hosted inside the 8x3.5" chassis? Please have a look at the photo of my server I have attached. Looks like it has two compartments for disks. Both are empty now, of course. Might it be the 16 bay chassis?

best regards,
Dmitry

 

[pimposh]

1) backplane power in ?
2) backplane pcb fixture plates ?
3) free PCI lanes to have enough bandwith to handle so many drives ?

If you really more than 4 NVMe drives i do not have answers to above questions. If 4 are fine, just get (since T640 support bifurcation if i am right, double check it) PCIe > NVMe card with or without PLX chip. Yes, they will be 2280/22100, not U.2/U.3.

 

[dmitry.n.medvedev]

1) backplane power in ?
2) backplane pcb fixture plates ?
3) free PCI lanes to have enough bandwith to handle so many drives ?

If you really more than 4 NVMe drives i do not have answers to above questions. If 4 are fine, just get (since T640 support bifurcation if i am right, double check it) PCIe > NVMe card with or without PLX chip. Yes, they will be 2280/22100, not U.2/U.3.

hi @pimposh , thank you for your prompt reply.

could you please explain to a newbie in layman terms what is "backplane power in" and "backplane pcb fixture plates"?
as far as the number of drives are concerned I haven't made my mind yet. If U.2 NVMEs are doable, then I would probably employ 4 to 6 of them. The rest of the bays could be used for sata/sas ssds, I guess.

 

[pimposh]

Just take a look how your existing 8xSAS/SATA backplane is powered and physically connected to backplane chassis. Then check if you have enough free power cables for second set. My bet is you don't, since configuration you want to have isn't factory one.

Another thing is NVMe or triple-mode NVMe/SATA/SAS. Guessing here isn't enough. Since i never had T640 with factory NVMe backplane i cannot deliver you solid answers, but both backplanes and HBA/RAID cards exists in variety of flavors.

Read carefully T640 service manual to find out what else might be a challenge. I'd start with PCIe lanes, if what you want really makes sense.

 

[ElCoyote_]

for the T640 with 8x3.5 bays, there is an actual NVMe conversion kit to add NVMe bays in the extra space above the 3.5 bays. Last I checked it was around 700-800USD, though:

Dell PE T640 8-bay SSD PCI-e HDD NVME Expander Backplane Kit [T640 Kit,PET640-NVMeCage-Kit] - R19,470.59 : Tekmart Africa, Data Centre Infrastructure Solutions

Tekmart Africa Dell PE T640 8-bay SSD PCI-e HDD NVME Expander Backplane Kit [T640 Kit,PET640-NVMeCage-Kit] - Dell Poweredge T640 8-bay SSD PCI-e HDD NVME Expander Backplane, Cage, and Cables Brand: Dell Specs:- Kit includes :: 8HDD NVMe Cage: PX42P :: NVMe BP PWR Cable: M16DC :: NVMe...

tekmart.co.za

 

[ElCoyote_]

Also found here: DELL POWEREDGE T640 8 BAY SSD PCIE HDD NVME EXPANDER BACKPLANE, CAGE, AND CABLES | eBay

 

[dmitry.n.medvedev]

Also found here: DELL POWEREDGE T640 8 BAY SSD PCIE HDD NVME EXPANDER BACKPLANE, CAGE, AND CABLES | eBay

ha! I talked to the seller and the seller is positive that this particular kit won't fit in the 8x3.5 chassis of mine I am confused who to trust, since it is quite pricey

 

[dmitry.n.medvedev]

I'd start with PCIe lanes, if what you want really makes sense.

oh... in fact I just want to convert this server to a fast developer machine. This is all I want. I don't want huge storage capacities or a fancy GPU. Just fast I/O, fast networking, FreeBSD + ZFS, a ton of RAM, and that is it there is to it.

do you think what I am trying to do, does not make a lot of sense? how would you approach the problem?

best regards,
Dmitry

 

[ElCoyote_]

ha! I talked to the seller and the seller is positive that this particular kit won't fit in the 8x3.5 chassis of mine I am confused who to trust, since it is quite pricey

This is interesting.. a friend of mine, who has the 18x3.5 bays T640 got told by the same seller (olinka) that the NVMe backplane only worked on the 8 x 3.5 chassis and 16x2.5 chassis. I am sure Olinka knows their stuff but the answer you got raises a few questions... What T640 chassis does their item work with if it doesn't with both the 18x3.5 and 8x3.5?

 

[ElCoyote_]

oh... in fact I just want to convert this server to a fast developer machine. This is all I want. I don't want huge storage capacities or a fancy GPU. Just fast I/O, fast networking, FreeBSD + ZFS, a ton of RAM, and that is it there is to it.

do you think what I am trying to do, does not make a lot of sense? how would you approach the problem?

best regards,
Dmitry

Makes a lot of sense but I'm not sure about FreeBSD on the machine.. I know for sure Linux works super great (and with ZoL) but you might want to check the FreeBSD compatibility list.

 

[pimposh]

ha! I talked to the seller and the seller is positive that this particular kit won't fit in the 8x3.5 chassis of mine I am confused who to trust, since it is quite pricey

So how anyone can call it an upgrade kit if it cannot be installed at all

This is interesting.. a friend of mine, who has the 18x3.5 bays T640 got told by the same seller (olinka) that the NVMe backplane only worked on the 8 x 3.5 chassis and 16x2.5 chassis. I am sure Olinka knows their stuff but the answer you got raises a few questions... What T640 chassis does their item work with if it doesn't with both the 18x3.5 and 8x3.5?

I'm unsure about T640 chassis, but if my memory ain't wrong, upper bays on Tx20 were typical 5,25" wide, while area where backplane with drive enclousures is fitted on bottom part - is bit wider in size. Might be the same in Tx40 and be the same reason why you just cannot put 2nd typical 8x3,5/16x2,5/8x2,5.U2 backplane/enclousure at top of chassis ? Since you got one, hope you can clear it out.

 

[dmitry.n.medvedev]

PS: dell community support suggested "you could order from our sales team a conversion kit for up to 8x PCIe SSD (NVMe)"

ref: Is there an NVME conversion kit for T640 8x3.5? | DELL Technologies

 

[dmitry.n.medvedev]

okay, it seems that no nvme conversion kit/nvme backplane for a Dell T640 8x3.5. damn it. fine.

maybe I will be able to source a 5.25 cage, put it above the 8x3.5 compartment, put 4 nvme drives in there and connect each one of them to a HBA330.

what do you think, gentlemen? will this setup work?

 

[dmitry.n.medvedev]

So how anyone can call it an upgrade kit if it cannot be installed at all



I'm unsure about T640 chassis, but if my memory ain't wrong, upper bays on Tx20 were typical 5,25" wide, while area where backplane with drive enclousures is fitted on bottom part - is bit wider in size. Might be the same in Tx40 and be the same reason why you just cannot put 2nd typical 8x3,5/16x2,5/8x2,5.U2 backplane/enclousure at top of chassis ? Since you got one, hope you can clear it out.

I believe you are talking about this ( please see the photo ). the top compartment differs in width from the bottom one just a tad, but the difference is visible.

 

[pimposh]

Above photo clearly shows little metal bracket that is used to bolt backplane which is missing on the top. This is what i meant by pcb fixture plate earlier on. And this is why i do still think that regardless what eBay seller says, you're not able to install it in regular 5,25" top bays.

BTW HBA330 is just SAS adapter therefore you can't match it with any NVMe drive. @ElCoyote_ path is fine and cheap and i second it.

If budget is not an issue - i'd get proper NVMe 5,25" bays from IcyDock, match it with proper NVMe PLX/HBA adapter if for some reason you want more NVMe drives. Easy as that.

 

[dmitry.n.medvedev]

Above photo clearly shows little metal bracket that is used to bolt backplane which is missing on the top. This is what i meant by pcb fixture plate earlier on. And this is why i do still think that regardless what eBay seller says, you're not able to install it in regular 5,25" top bays.

BTW HBA330 is just SAS adapter therefore you can't match it with any NVMe drive. @ElCoyote_ path is fine and cheap and i second it.

If budget is not an issue - i'd get proper NVMe 5,25" bays from IcyDock, match it with proper NVMe PLX/HBA adapter if for some reason you want more NVMe drives. Easy as that.

ah, now I think I understand the pcb fixture issue. thank you!

I am going to give a thought to what @ElCoyote_ suggested.

PS: also, many thanks for pointing out info regarding the HBA330!

 

[dmitry.n.medvedev]

Above photo clearly shows little metal bracket that is used to bolt backplane which is missing on the top. This is what i meant by pcb fixture plate earlier on. And this is why i do still think that regardless what eBay seller says, you're not able to install it in regular 5,25" top bays.

BTW HBA330 is just SAS adapter therefore you can't match it with any NVMe drive. @ElCoyote_ path is fine and cheap and i second it.

If budget is not an issue - i'd get proper NVMe 5,25" bays from IcyDock, match it with proper NVMe PLX/HBA adapter if for some reason you want more NVMe drives. Easy as that.

okay, here is the plan:
2. first of all I will install the HBA330 and start buying SSDs ( not sure which exactly, 6g? 12g? )
1. install the FreeBSD onto the BOSS s1 and check if it is usable w/ the hardware
3. in some future buy NVMe adapter and start buying disks. one by one.

let's see how it goes.

BTW, many warm thanks to each and every one who offered his knowledge on the subject! without you I will still be hunting for a conversion kit and trying to connect HBA330 to an SSD.

best regards,
Dmitry

 

[ElCoyote_]

Personally speaking and not wanting to dump 800USD on a backplane, I went with the U.3 carriers from Linkreal for my Micron 9400 drives. This allowed me to repurpose the backplane budget and get better drives (a few Micron 9400 15Tb). I also kept the SN260 7.68Tb HHHL cards (two per hypervisor). This occupies a total of 3 PCIe slots just for NVMe but since the T640 has so many, it wasn't a real problem (I also have an X710-DA4, a single-slot GPU, an H750P for the SAS backplane and a Sonnet card for extra USB-C ports). (and I think I still have one free PCIe slot)