Mellanox Innova2 CONNECT-X 5 25gbps SFP28 and XILINX Kintex ULTRASCALE+ DPU $250 + Bestoffer

applepi

Member
Jun 15, 2013
52
26
18



Accepted $150 each for 2 of them, only issue programming these is that it requires a Vivado License to put custom HDL on the FPGA, using it as a NIC should still work.
 

applepi

Member
Jun 15, 2013
52
26
18
Okay just got these cards today, It seems they just had no firmware installed on the device, I flashed the latest firmware availble through the mellanox site and reflashed using the recovery options in flint similar to how the connectx-4 cards were with blank firmware. After flashing I rebooted and ran mlxupdate to update to the latest firmware and then set the GUID and MAC to arbitrary values that are different from the other NIC I have.

I have not yet flash the FPGA on both NICs but one appeared to be flashed (based off the LED status)

Once I install linux on my test lab computer I should be able test flashing the default FPGA firmware and testing out these NICs.

PS. I also found some ADM-PCIE-8V3 on ebay for $50 which are a bit older FPGA but have QSFP28 transceivers connected directly to the FPGA
 

applepi

Member
Jun 15, 2013
52
26
18
Okay, just a heads up to anyone trying to get these cards working, DO NOT flash the Lenovo firmware or firmware in the bundle. It will secure-fw your card and you will no longer be able to flash unsigned firmware to the card. I got around this by using a CH341A mini programmer and a russian CH341 programmer software to flash the Winbond 128MBIT WSQ128 SPI flash to backup and wipe the firmware and then I could have the card boot into frimware recovery mode and flash unsigned firmware.
 

americanfuzzylop

New Member
Mar 16, 2021
3
0
1
Bummer, mine does not come with a cooler. Would you mind taking a picture of yours? Thanks!

wouldn't keep timing once programmed
Did you mean there's timing violation of the logic programmed to the FPGA?

By the way, I saw a lot of folks (100+) purchased the card. Am wondering if there's an online mailing list, etc. to discuss experience playing with this card.
 

anything1233

A New Member
Dec 29, 2020
33
12
8
@americanfuzzylop Here’s mine (clickable):
3A35245D-202F-441B-85A6-35E11AEAD049.jpeg
Ordered 3x, haven’t tested yet. But even with the meticulous packaging and a mile of adhesive tape, one arrived with a bent PCB, chewed up OpenCAPI connector, and a stuck fan. But it’s fixable. Though that timing issue is a little intimidating... @applepi please keep us posted
 
Last edited:

americanfuzzylop

New Member
Mar 16, 2021
3
0
1
But even with the “meticulous” packaging, one arrived with a bent PCB, chewed up OpenCAPI connector, and because of that the cooler has stuck as well
That's too bad... sorry to hear that :(

And thanks for the photo! I guess I will just stick two heat sinks to the FPGA and the Eth PHY.

What do you plan to do with the FPGA? It's a pretty decent chip. I do have Vivado licenses (have not checked if this particular chip is covered, but all Ultrascale+ chips should be covered), but have not decided what to do.
 

aleruggeri87

New Member
Feb 20, 2022
1
0
1
Hi everybody,
I also saw the board a few weeks ago on ebay, and looking for additional info I found this thread.
Anyone of the guys who has bought the board (@applepi, @anything1233, @americanfuzzylop) has some update to share?
I'm a little bit worried for what @applepi said regarding the timing; on the pictures on ebay I can see that a few decoupling capacitors are missing, what is the situation on your boards?

Thanks and regards,
Alessandro.
 

Cute_SC2

New Member
Mar 7, 2022
4
0
1
I bought some 2 years ago, the boards are sold broken, only the fpgas themself are working and needed to be soldered onto another working board.
There is a JTAG header on the top left side of the board that can be used for programming, also it can be programmed over pcie with the mellanox tools.
For simple function testing without loaded bitstream the heatsink can be skipped.
 

mwrnd

New Member
Jul 5, 2022
3
11
3
I just came across this forum. I have been working on detailed notes for my Innova-2 for a while.

These boards are being sold with no guarantees but with some effort they can be refurbished and make excellent FPGA Development Boards. I was able to fix capacitor, JTAG, and DDR4 Byte Lane errors and now have a mostly functional board.

@Cute_SC2 - Please do not skip the heatsink. Running the Flex Image the FPGA gets above 70C. At idle with no SFP cables/modules the ConnectX-5 IC on my board overheats within minutes and its SFP status LEDs go red. It gets hot enough to burn skin.

@aleruggeri87 - When I slow the memory down to DDR4-1600 my designs meet timing. Better cooling also seems to help.

@applepi - Consider a thorough wash of your board in 99% Isopropyl Alcohol in an Ultrasonic Cleaner to remove any dust under the BGA ICs. Also, try improving your cooling solution. Larger heatsink and a server blower fan aimed directly at the board.

@anything1233 - "one arrived with a bent PCB" - Do you think it can be salvaged? If you or anyone else has a truly unsalvageable board and are planning to desolder the ICs, please consider spending some time to buzz out the IC-to-TestPoint connections with a multimeter.

These boards are too capable to scrap. The XCKU15P has 1M+ Logic Cells. Closest retail product would be the Xilinx Alveo U25.

If the power supply works it is worthwhile to try and fix your board.
 

bayleyw

Active Member
Jan 8, 2014
163
50
28
I just came across this forum. I have been working on detailed notes for my Innova-2 for a while.

These boards are being sold with no guarantees but with some effort they can be refurbished and make excellent FPGA Development Boards. I was able to fix capacitor, JTAG, and DDR4 Byte Lane errors and now have a mostly functional board.

@Cute_SC2 - Please do not skip the heatsink. Running the Flex Image the FPGA gets above 70C. At idle with no SFP cables/modules the ConnectX-5 IC on my board overheats within minutes and its SFP status LEDs go red. It gets hot enough to burn skin.

@aleruggeri87 - When I slow the memory down to DDR4-1600 my designs meet timing. Better cooling also seems to help.

@applepi - Consider a thorough wash of your board in 99% Isopropyl Alcohol in an Ultrasonic Cleaner to remove any dust under the BGA ICs. Also, try improving your cooling solution. Larger heatsink and a server blower fan aimed directly at the board.

@anything1233 - "one arrived with a bent PCB" - Do you think it can be salvaged? If you or anyone else has a truly unsalvageable board and are planning to desolder the ICs, please consider spending some time to buzz out the IC-to-TestPoint connections with a multimeter.

These boards are too capable to scrap. The XCKU15P has 1M+ Logic Cells. Closest retail product would be the Xilinx Alveo U25.

If the power supply works it is worthwhile to try and fix your board.
Wow, that's an incredible amount of documentation. Thanks!
 
  • Like
Reactions: linuxmanbg

anything1233

A New Member
Dec 29, 2020
33
12
8
@mwrnd the docs you‘ve made are pure gold! The sheer amount of effort you dedicated for this is simply mind-boggling! Thank you!

Do you think it can be salvaged?
The bent part mainly affected power delivery part, and while I haven’t tested the FPGA itself yet, ConnectX-5 does work just fine. Just had to fix the cooler.

I was able to fix capacitor
Which one?
 
Last edited:
  • Like
Reactions: linuxmanbg