NVIDIA BlueField-2 200GbE DPUs for under $469

[MiniKnight]

Here's the link: NVIDIA BlueField-2 Ethernet DPU200GbE Model number:BF2M515A | eBay

Looks like they're BF2M515A-VENOT

From what I saw searching, they're ES cards, and have only Port 1 active.

They're $469 for one, but if you're at four, you're at $422 each. That's one of the cheaper 200GbE options out there. Even if you're not using Arm, it's just a ConnectX-6 full-height.

 

[wifiholic]

I bought one of these from this seller last fall. For what it's worth, they can be cross flashed to the 516A (IIRC) 2x100G model.

 

[MiniKnight]

I bought one of these from this seller last fall. For what it's worth, they can be cross flashed to the 516A (IIRC) 2x100G model.

Oh wow, so you've now got two 100G ports on it? Was it hard?

 

[wifiholic]

Oh wow, so you've now got two 100G ports on it? Was it hard?

Yes, I have two 100G ports now on a BF2M515A. It was very difficult because I didn't know what I was doing, so I stumbled through it and didn't document the steps well enough in the moment. A few days later I tried to write up the process from memory, but that memory had already gotten pretty fuzzy, so I'm hesitant to share for fear of enabling someone else to brick their Bluefield...

In short, however, I recall having to bridge the JP1 (FLASH_PRSNT) jumper to be able to change the PSID, and I used flint with -allow_psid_change to flash firmware from the MBF2M516A. That's all I can say for certain, I'm sorry.

Edit: I thought I'd checked if both ports actually worked after I flashed it, but I checked now just in case, and after moving the link from one port to the other, it worked fine.

P.P.S. I trust that most of you all know this, but for anyone new to these type of NICs, if you're going to use one in a desktop, I strongly recommend rigging up a fan in the PCIe slot bay below it. They get hot if they don't have enough airflow.

 

[Civiloid]

I have that card for quiet some time, however I didn't thought about posting my experience with it on a forum for some reason.

Basically what previous person said - I've cross-flashed it as well to MBF2M516A and it works in dual-port mode ever since then. I think I just used flint with -allow_psid_change.

It gave me a small heart attack as after first reboot it took about 15 minutes(!) for the PC to boot. Reason was - extremely old SDK that was flashed into it (something ubuntu 14.04 based I think). It was constantly complaining about eMMC I/O errors and various other problems, but after I've flashed recent SDK (took very long time, but I've used suggested nvidia procedure) it started to work just fine. However for my tests I needed just the NIC part so I've disabled the DPU part of it as it was adding a bit of extra latency on top.
I'm not sure though if it would be better to flash a new SDK before cross-flashing.

And yes, ConnectX-6 gets extremely hot if both ports are used, and BlueField-2 is not an exception here (it is actually slightly hotter). Without a forced airflow (high static pressure or at least a PC fan that is very close to the board and blows along side the fins of the radiator) it will actually thermally shutdown due to overheat and you would be able to burn yourself if you are not careful enough.

P.S. I'm not sure it is an ES card, it is probably just a custom-made card with custom firmware that was produced for specific customer.

 

[darrian157]

This is a long shot, but do either of you who cross flashed this card remember which firmware you grabbed? there are 6 different versions of MBF2M516A on the nvidia site.

 

[Civiloid]

I've grabbed MBF2M516A-CENOT one, because it is E version of a board (P version have additional power connector and have higher clocks on a CPU side). I wasn't sure if crypto was enabled or not as it showed conflicting information in various utils. So I went for crypto disabled one.
Probably crypto enabled should also work and i'd try that one first.