PCIE to U.2 SFF card compatibility with Cisco C220 M4

[Kandrew]

Good Morning

I have a few Cisco C220 M4 8-Bay servers that I am trying to get a U.2 NVME drives working on. I have been able to update the BIOS, but have had no luck with getting a 10GTek SFF 8643 PCIe card to be recognized by the machine. I am using the Rev A riser and have tried both slots. In the other slot, I have a 10G SFP+ card that is recognized and works. I have confirmed the card and 2TB Intel U.2 Drive does work in another machine.

I know of the Rev B riser that, I believe, adds the 2 SFF-8087 ports to the riser, so the server can support it natively. However, I can not find them for sale anywhere, and rather not go down that road anyway. A PCIe card will work fine for me.

My issue, as stated, is I can NOT get the PCIe card to be recognized and/or loaded. The wording in the User-Guide/Manual/Spec-Sheet for the server makes it seem that the NVME functionality may ONLY be unlocked with that REV B riser. Does anyone know if this is true, or if any PCIe should work? Has anyone had any experience with this? I have searched through the BIOS for settings, but can't find anything that'll work. I do NOT need bifurcation support. As a side note, I do have both CPU Sockets populated with Intel E5-2697 v4.

Thank You in Advance

 

[nexox]

It depends on the adapter card, but the cheap ones are mostly passive and won't show up as a device on their own, were you testing with the U.2 drive connected?

 

[Kandrew]

It depends on the adapter card, but the cheap ones are mostly passive and won't show up as a device on their own, were you testing with the U.2 drive connected?

I was yes. The bios has a section specifically for checking PCIE connectivity, which yeah since they're passive cards, might not show. The server has 2 bays that have U.2 capable plugs and an accompanying SFF-8087 plug on the Backplane. I used both a USB-Boot Linux instance to check for the drive/PCIE card with LSPCI command, and the Proxmox instance I have installed on 2 SSDs in other Drive Bays, but neither show the U.2 Drive.

These drives are older Intel P4510 2TB. Cisco has similar drives they sold that are made by Intel (well partnered) that also use the same NVME 1.2 and Gen 3 interface. The only thing I haven't tried is the second U.2 capable bay. I'm fairly certain that's not the case, but will update if it works lol.

 

[nexox]

It could be that the cisco backplane doesn't use a standard pinout, especially since SFF-8087 is somewhat uncommon for NVMe, can you test with an SFF-8643 to SFF-8639 cable directly to the drive?

 

[Kandrew]

I can see what I can try. I do have a 8639 adapter. Something to note, in Cisco's paperwork, they constantly mention the "Riser and Cable kit must be purchase/is needed to add this functionality". In other words...you may be right. Might be able to check continuity of cable and backplane for the server to verify. If that's the case...these bad boys will be for sale lol.

I assumed this was only to add "NATIVE" functionality.....

 

[Kandrew]

So quick update:

As far as I can tell, the necessary connections for the use of NVME drives are not present on the Riser Card PCIE slots electronically.

The Rev B riser includes 2 SFF-8087 ports which add the functionality for the U.2 NVME drives. I believe Cisco designed their Rev A riser to exclude these connections, so I am NOT able to get a 3rd Party (10GTek PCIe SFF Card) simple conversion card working. The BIOS is not able to recognize the card, so it does NOT get loaded.

In the documentation, they mention the "Requirement" of the Rev B riser to have NVME compatibility. I assumed this was for Native support and kept the 2 PCIe slots on the riser available, but still left the ability for a user to install a PCIe SFF card like I tried. The server (released in Sept 2014) supports up to NVME Rev 1.2 (Released in Nov 2014) with 2 U.2 interfaces (released in Dec 2011). Being an industry stable, I can assume they had access to information for the 1.2 rev of NVME well before release and so was able to work it into their server platform. Probably with little issue, because I see little changes with the new revision.

In the spec sheet for the server, Cisco lists the PCIe ports on the board to be 2 x24 connections, that then convert to 2 x16 slots (electronically) on the risers basically. My guess is the connections used by NVME are ONLY routed to the 2 SFF-8087 ports on the Rev B riser. I am assuming this was done to add the 2 x4 lanes needed for the 2 NVME bays upfront and keep x16 support for the normal PCIe slot on the riser.

I am by no means an expert, and maybe all this would be apparent in the spec sheets if the reader isn't completely r-word'ed. However, this feels like an artificial limitation. Only having those connections present on the Rev B riser, and not also wired for compatibility on the Rev A seems like a complete dick move. It sucks, because the M4 platform is great. CIMC is fantastic, they're 1U, cheap, the supporting hardware cheap, and have great expand-ability.

Obviously this platform is old and way out-of-date, but I feel like I wasted a month of work(or more) trying to figure out why my PCIe cards weren't being recognized. I feel they could have added a line in the spec sheet saying something along the lines that the Rev A riser does NOT have the proper connections for the NVME Bays up front, and so, even 3rd party cards will NOT work. Like I said earlier, maybe people smarter than me would have realized this immediately. I searched online for posts about similar issues, but found none. Hopefully this post helps someone going forward.

As per my last post, the installation guide had the info I needed, NOT the spec sheet. The spec sheet was really the only document I referenced before purchasing the servers last year, hoping to upgrade to NVMe in the future.

Thank You

 

[nexox]

There should be nothing special about those NVMe connectors on the riser except the pinout, PCIe lanes are PCIe lanes and unless the BIOS is really locked down NVMe should work in any slot (particularly since Intel was selling AIC SSDs at that time and I'm pretty sure Cisco offered them in servers.) Still unless you can find the riser and cables you likely can't use those U.2 bays without reverse-engineering the pinout and building custom cables (no idea how to go about that.)