Beware of EMC switches sold as Mellanox SX6XXX on eBay

Thekruk

New Member
Apr 16, 2019
1
0
1
Hey, I'm looking for the conversion guide, but have not received a reply from Mpogr just like Klui.
Any idea when hes back? Or any chance someone else could share the guide? Thanks!

Cheers!
 

Zhenfang Wei

New Member
Apr 25, 2019
3
0
1
You also an flash directly on the switch, i (re-) posted the instructions several times in this thread:)
Hi,Rand__
May I ask how to flash the firmware directly on the switch. I got stuck at the very last step of step 6. I can boot into minilinux and EMCOS but I don't know where can I start the flashing.
 

Antonis Papadakis

New Member
May 12, 2019
4
0
1
I have bought a Mellanox FDR Switch SX60151F-1SFS which is suppose to support FRD. However, i was connecting to FDR10 hence i tried to update the driver and found the following warning: The PSID on flash (EMC1260110029) differs from the PSID in the given image (MT_1260110029). Two issues arise: a) The PSID 1260110029 refers to FDR10 and not FDR switch SX60151T-1SFS. Can the switch support FDR (i.e. upgrade the PSID to 1260110020 which is the FDR model?) since the firmware installed is SX60151T-1SFS which only supports FDR10 and not SX60151F-1SFS which supports FDR (i) b) If it does, how i will move from EMC to MT version?? Thanks.
 

Rand__

Well-Known Member
Mar 6, 2014
4,577
911
113
I have bought a Mellanox FDR Switch SX60151F-1SFS which is suppose to support FRD. However, i was connecting to FDR10 hence i tried to update the driver and found the following warning: The PSID on flash (EMC1260110029) differs from the PSID in the given image (MT_1260110029). Two issues arise: a) The PSID 1260110029 refers to FDR10 and not FDR switch SX60151T-1SFS. Can the switch support FDR (i.e. upgrade the PSID to 1260110020 which is the FDR model?) since the firmware installed is SX60151T-1SFS which only supports FDR10 and not SX60151F-1SFS which supports FDR (i) b) If it does, how i will move from EMC to MT version?? Thanks.
Most likely it should work - and since the 6015 does not have a Subnet Manager (and management gui and all the other benefits of the managed switches) the conversion might be done with just the crossflash, but i don't know for sure, dont have a non managed one.
 

Zhenfang Wei

New Member
Apr 25, 2019
3
0
1
Hi, Rand__, I really appreciate your help. Unfortunately I got stuck at the step 6 (flash the firmware before the 1st run). Thus I don't have a MLXOS which can boot correctly on my switch yet. All I have is EMCOS, minilinux and the file system of the full MLXOS.
I googled around and seems no other person got the same issue as me (Unable to query the switch info via flint command even the FDR IB link is up). I am not sure if it is the problem of the Switch, the DAC cable or the network adapter..
 

Rand__

Well-Known Member
Mar 6, 2014
4,577
911
113
I am not sure if the MLX OS at that point is not a misnomer - if you have flint on your EMC OS then just flash the firmware with it from there.
 

Antonis Papadakis

New Member
May 12, 2019
4
0
1
Most likely it should work - and since the 6015 does not have a Subnet Manager (and management gui and all the other benefits of the managed switches) the conversion might be done with just the crossflash, but i don't know for sure, dont have a non managed one.
You mean that even though it has a firmware that shows that does not support FDR, if i change the firmware to FDR compatible, the switch will work as FDR. By the way, there is no way of telling what type of switch it is? Thanks for the prompt response.
 

Rand__

Well-Known Member
Mar 6, 2014
4,577
911
113
well the crossflashing is being done all the time on cx3 nics, so i'd assume the same would work on the switch.
Also, there used to be a license based upgrade from 40 to 56 GB/s, so it can't be a hardware limitation
 

Antonis Papadakis

New Member
May 12, 2019
4
0
1
well the crossflashing is being done all the time on cx3 nics, so i'd assume the same would work on the switch.
Also, there used to be a license based upgrade from 40 to 56 GB/s, so it can't be a hardware limitation
I am a bit confused on what is crossflashing. If you mean using mlxburn or flint, i tried them and do not work. It says the PSID is different and aborts. Thanks.
 

Rand__

Well-Known Member
Mar 6, 2014
4,577
911
113
crossflashing is the process of changing from one psid to another (for mlx cards).
There is a flint parameter ('--allow-psid-change' ) that prevents the abort. O/C if the firmware file is not compatible you might brick your switch (worst case scenario)
 

Labs

Member
Mar 21, 2019
46
7
8
For those who want to flash the ASIC firmware directly on the switch the following procedure I see as the simplest one.
Interrupt the boot process and go to uboot prompt:
Code:
setenv extra_args single
boot
When you see the shell prompt configure the network first based on your topology.
In example below 192.168.0.1 is the switch address and 192.168.0.100 is the tftp server address.
Code:
ifconfig eth0 192.168.0.1 netmask 255.255.255.0
cd /tmp
tftp -v 192.168.0.100 -c get MSX6012.bin
mst start
flint -allow_psid_change -override_cache_replacement -d /dev/mst/mt51000_pciconf0 -i MSX6012.bin burn
If you used the archive from user JSLEnterprises, MSX6012.bin file is the correct one you need to use.
Exit shell and the switch will continue to boot.
 

Labs

Member
Mar 21, 2019
46
7
8
Sx6012 or 6018 ,.
If it is 6012 only one mgmt port , why two Mac ?
If it is 6018 then wrong firmware
All SX switches have 2 management interfaces. SX6012 has only mgmt0 connected to the external port since there is no space to install another one due to small size. The other SX switches have 3 RJ45 ports 2 for management and one for serial connection. I am talking about managed switches only.
 

Labs

Member
Mar 21, 2019
46
7
8
Flashed to MT_1270110020, shortened the shared secret string to 48 char... touched the firstboot and firstmfg flags again, no dice :(
I checked some logs and I compared it with yours and I saw this part missing in between:
Code:
Writes a grub.conf which use the selected next boot location,
   and which contains the installed image version strings.

Generating SSH1 RSA host key: [  OK  ]
Generating SSH2 RSA host key: [  OK  ]
Generating SSH2 DSA host key: [  OK  ]
 
Starting sx_low_level_if:
Not sure which part is triggering that but maybe you can investigate there.
One question: Did you format the flash like in the guide? You have more nand errors than usual but maybe I am wrong.

After you see the part below you should not see any nand errors but I see the errors in your case are still coming...

Code:
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
[  OK  ]
 

Terry Wallace

PsyOps SysOp
Aug 13, 2018
138
79
28
52
Allen, TX
You DO need to get the matching firmware for your switch model.
The 6012.bin is for the sx6012 12 port version only.
You have to hex cut the 6018 or 6036 models yourself as documented.
 

Skuzzi

New Member
Sep 20, 2018
5
1
1
For those who want to flash the ASIC firmware directly on the switch the following procedure I see as the simplest one.
Interrupt the boot process and go to uboot prompt:
Code:
setenv extra_args single
boot
When you see the shell prompt configure the network first based on your topology.
In example below 192.168.0.1 is the switch address and 192.168.0.100 is the tftp server address.
Code:
ifconfig eth0 192.168.0.1 netmask 255.255.255.0
cd /tmp
tftp -v 192.168.0.100 -c get MSX6012.bin
mst start
flint -allow_psid_change -override_cache_replacement -d /dev/mst/mt51000_pciconf0 -i MSX6012.bin burn
If you used the archive from user JSLEnterprises, MSX6012.bin file is the correct one you need to use.
Exit shell and the switch will continue to boot.
This doesn't work for me. I get command not found when I type basically anything, like cd /tmp, flint etc.

However in this boot mode the IB net does come up, and I can run ibswitches from a client connected to IB, but I can't burn the firmware image, I get "cr read access to lid 2 failed", guessing its the switch blocking it. Not sure where to go from here. When in mlxlinux the port's dont come up at all.
 

Labs

Member
Mar 21, 2019
46
7
8
This doesn't work for me. I get command not found when I type basically anything, like cd /tmp, flint etc.

However in this boot mode the IB net does come up, and I can run ibswitches from a client connected to IB, but I can't burn the firmware image, I get "cr read access to lid 2 failed", guessing its the switch blocking it. Not sure where to go from here. When in mlxlinux the port's dont come up at all.
If you get command not found it means something is wrong with your switch OS. You run these commands after the conversion is completely done. This is the last step. Do you have a standard EMC SX6012? Or are we talking about different switch?