Beware of EMC switches sold as Mellanox SX6XXX on eBay

meltman

New Member
Jan 29, 2017
1
0
1
36
I want to recognize mpogr for his excellent conversion guide.

I'm now the proud owner of a fully functional Mellanox SX6012 switch.

A bit more info about the switch - once it's fully up on MLNX-OS it is quite tame noise wise. The fans are more than insane while booting or outside of a fan controlled environment.

The switch has active power management and reporting. My switch is reporting between 30-40w energy use with four 10Gb SFP+ modules in QSFP+ adapters.

I'm only using it as an L2 ethernet switch as of now.

Cheers!
 

Homegamer

New Member
Feb 11, 2018
1
0
1
25
@mpogr Hey, do you still have the guide for flashing the MLNX-OS?
Would be great if you could help me with the conversion.

I sent you a PM!
 

mpogr

Active Member
Jul 14, 2016
113
81
28
50
Guys, I'm on a holiday (in The Bahamas :) ) and have just received a tremendous amount of PMs asking for the guide. Please, be patient, I'll answer each and every one of you when I have time. Cheers!
 
  • Like
Reactions: _alex and Marsh

llowrey

Member
Feb 26, 2018
88
60
18
I am eyeing an SX6018 and am hoping @mpogr's procedure will work. But, worst case, if the EMC SX6012 can be flashed with the SX6005 firmware for unmanaged use does that mean that the SX6018 should be able to be flashed with the SX6015 firmware? So, at worst, I end up running opensm somewhere and have an IB only network using the SX6018. Correct?
 

mpogr

Active Member
Jul 14, 2016
113
81
28
50
I am eyeing an SX6018 and am hoping @mpogr's procedure will work. But, worst case, if the EMC SX6012 can be flashed with the SX6005 firmware for unmanaged use does that mean that the SX6018 should be able to be flashed with the SX6015 firmware? So, at worst, I end up running opensm somewhere and have an IB only network using the SX6018. Correct?
All correct, however, I don't see any reason why you wouldn't be able to convert it to proper SX6018. A few people have already done that.
 

llowrey

Member
Feb 26, 2018
88
60
18
I can report another successful conversion, thanks to @mpogr. I now have a fully functional SX6018. I'm not doing anything fancy atm (no infiniband) but am exercising a few interesting bits of functionality.

I have one CX3 connected via a no-name DAC. The switch allows me to select 56GbE but I get no link when I do. From what I understand, my DAC is probably coded for 40Gb max so I need to chase down a DAC rated for 56Gb (FDR).

I have a Dell branded Avago transceiver hooked up to 4 different hosts via an MPO->4xLC breakout cable from FS and a no-name QSFP+->4xSFP+ DAC breakout going to 4 hosts. Splitting the ports had to be done via the CLI but the split ports can be managed in the web UI.

I'm also running several VLANs and all pass through to my other switch just fine.
 
  • Like
Reactions: Mithril and _alex

llowrey

Member
Feb 26, 2018
88
60
18
FYI, I finally acquired an FDR cable (Mellanox) and am now able to connect at 56GbE. I only have one 56GbE device so I can't verify the speed but both the switch and card show a link speed of 56GbE.
 

llowrey

Member
Feb 26, 2018
88
60
18
I seem to have a problem with setting the LAG hash mode. According to the manual, more than one criteria can be specified. But, when I check more than one box on the web UI only last one remains checked after the change is applied. The same thing happens when I do the same via the cli. Bug?

Code:
ibswitch [standalone: master] (config) # port-channel load-balance ethernet source-destination-ip source-destination-port
ibswitch [standalone: master] (config) # show interfaces port-channel load-balance
source-destination-port
ibswitch [standalone: master] (config) #
 

jak

New Member
Apr 17, 2016
9
3
3
First I would like to thank @mpogr for conversion guide.

I currently have two SX6012 connected together with a Mellanox FDR cable. The link between the two switches does not come up if I set the speed to 56 GbE through the web interface or when I force the interface speed in the command line. This is all done in the ethernet profile. Am I doing something wrong?
 

jak

New Member
Apr 17, 2016
9
3
3
Both switches have the 56 GbE license active and they are connected over an FDR cable.


 

llowrey

Member
Feb 26, 2018
88
60
18
I've noticed that, for my one linux box that has a 56GbE card (CX3), it doesn't show link until about 60s after my 1GbE NIC does. It's solid once it's up but there seems to be a delay that I don't get when the port is set to 40GbE.
 

thfs1002

New Member
Mar 24, 2018
2
0
1
52
Hi, all.

As I just simply can't find the SX6012 firmware, so I'd like to flash EMC6012 into MLNX 6005.

So, I'd like to flash the firmware into my emc sx6012 by issuing below:

flint –-allow_psid_change –d lid-0x<yourswitchlid> -i ./<firmwarename>.bin b

Where firmwarename is fw-sx-x_y_zzzz-MSX6005F_Ax.bin. Is it ok?
 

jak

New Member
Apr 17, 2016
9
3
3
The server I am using has a Mellanox ConnectX 3 adapter with the OFED driver on ubuntu 16.04. Connecting to the switch in ethernet mode exhibits will only connect using 40 GbE and waiting does not help. Connecting to the switch in VPI or IB mode using infiniband it will only connect at FDR10 speeds. When I look at the switch all the ports say their supported speed maxs out at FDR10 instead of FDR. The switch will not allow me to force FDR to be a supported speed on any of the ports.
 

mpogr

Active Member
Jul 14, 2016
113
81
28
50
The server I am using has a Mellanox ConnectX 3 adapter with the OFED driver on ubuntu 16.04. Connecting to the switch in ethernet mode exhibits will only connect using 40 GbE and waiting does not help. Connecting to the switch in VPI or IB mode using infiniband it will only connect at FDR10 speeds. When I look at the switch all the ports say their supported speed maxs out at FDR10 instead of FDR. The switch will not allow me to force FDR to be a supported speed on any of the ports.
Did you try forcing the FDR speed using the iboportstate trick described in the guide (in IB mode)?

Sent from my SM-G935F using Tapatalk