Odd issue with X11SSH-f and Mellanox CX3-EN

PigLover

Moderator
Jan 26, 2011
3,081
1,388
113
I was trying to duplicate a build inspired the "Low Power Ceph Node" by @niekbergboer (Low-power Ceph/ProxMox node).

Using the same CPU (i3-6100T) but a slightly different MB (X11SSH-f) and a Mellanox CX3-EN 10Gbe nic.

The Motherboard is flashed with BIOS 2.0.

No matter what I do the system won't post with the CX3 card installed.

  • With everything normal and BIOS defaults it hangs at "B2 - loading peripheral legacy OPROM"
  • If I disable loading the option prom on the Mellanox card it gets past taht but stalls at "99 - i dont remember..."
  • If I remove the CX3 card it posts normally
  • If I install a X520-DA2 it posts normally (but I can't use it because the @!#%%#$! Ubiquiti switch won't support longer DAC cables and I don't have legit Intel SFP+...
I know the CX3 card is perfect - plus I have several and have swapped in known working cards. Considering just ordering a Chelsio card because it will probably work and isn't locked to specific SFPs. I don't really have time to work with SM to debug it...

Just odd...
 

whitey

Moderator
Jun 30, 2014
2,771
872
113
40
I know you had said you cobbled together 'some' working DAC's and maybe if memory serves me correct you also had some optics/fiber that you had used on your Ubi 10G Switch. I know it's not the answer you were looking for but I can't imagine that ordering proper Intel LR optic and using fiber would be cheaper than buying a Chelsio unless you found one that 'fell off the truck' :-D

FWIW here is the fs.com listing for the proper Intel optic that will work w/ those X520-da2's.

Intel E10GSFPSR 1000BASE-SX and 10GBASE-SR SFP+ Transceiver | FS.COM

GL w/ build, I DO love me the idea of super low pwr CEPH nodes.
 

PigLover

Moderator
Jan 26, 2011
3,081
1,388
113
Only 1 meter DACs are working with the switch. This one is living at the bottom of the rack so 1M DAC won't do. I have other reasons to hate the X520. If I do spend money it will be on a Chelsio card - if this experiment doesn't pan out the way i want the board will be moving to my FreeNAS host and freeing up a Xeon-D D-1518 board and the Chelsio cards are better supported on FreeBSD.

If I can get my hands on the 1.1 BIOS I'm considering flashing back to that to see if it has any effect.
 

whitey

Moderator
Jun 30, 2014
2,771
872
113
40
Hey you ain't gotta tell me abt the hate for X520's (love and hate for me), I just got rid of three of em' for MLX CX3 VPI's (HP branded crossflashed back to MCX354A FW/.bin)

Cool deal, press on!
 

PigLover

Moderator
Jan 26, 2011
3,081
1,388
113
I opened a ticket with SM. Tried a few things - updated the Mellanox card to current firmware (release 10 Jan 17). Tried a few BIOS settings. No love - still hung during POST if the Mellanox card was installed.

SM send me the prior BIOS (rev 1.1). Downgraded the BIOS and viola...all is well.

Now somehow I just have to remember not to update the BIOS someday in the future.
 

gzorn

Member
Jan 10, 2017
76
14
8
@PigLover - I just saw your ref to this in another thread (sorry to necro this). I've got a Supermicro X11SSI-LN4F that I had updated to BIOS 2.0. I recently got a Connect-X3 card (dual 10G-SFP+). It also refused to POST as well.

There's now a BIOS rev 2.0a for my motherboard (released in March). Updating to that seemed to fix the issue - it now POSTs and recognizes the Mellanox card. There seems to also be a 2.0a for your motherboard as well. It might fix your situation and update you.
Good luck.
 
Last edited:
  • Like
Reactions: PigLover