Ha wow, the irony with that unit is the system I have has two of those sockets directly adjacent to the SFP+ ports, and my desktop has an Aquantia controller built in, which was half the reason for this system.
Tried the patch cable with Intel and Finisar units. One of the Intel units is a branded Finisar module.
I am already in contact with Supermicro, and I'm pushing them to check the proper SFP+ units and DACs.
So after swapping out the Mellanox for the X520 I still am having no luck. The new card establishes a link over fiber with the Mellanox in my desktop without issue, but the X553 adapter refuses to link up with anything other than the other X553 in the same machine.
Guess it's time to contact...
Neither NIC supports autonegotiation, with the only difference I've seen in ethtool being the use of pause frames.
I have an X520 card on the way, I'll swap out the Mellanox and see if that doesn't resolve things.
Firmware is up to date per Mellanox's installer package (2.9.1000), but there's a newer 2.9.1200 revision on the Mellanox site and I'm trying to get that installed manually since their tool won't do it.
OS is Ubuntu 16.04 server.
Now that I test with mstconfig apparently mellanox has dropped...
I've been working on a Supermicro SYS-E300-9A (Intel Atom C3858,) putting together a fully virtualized router + whatever, and I'm getting around to being complete. The last step for me is routing my existing 10gbit setup through it, but I've hit a snag.
My NAS has a Mellanox ConnectX-2 card...