Beware of EMC switches sold as Mellanox SX6XXX on eBay

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

RageBone

Active Member
Jul 11, 2017
617
159
43
AFAIR there were images for the SX6018 in the hex editing part of the guide.
What? Sorry but i don't understand how or where you mean.
I have the PDF and docx versions of the guide, the later from the JS.. archive, in which there is a prepared binary for the sx6012 but not the 6018 and i haven't grasped that those would be compatible.
The docx or pdf itself don't contain any firmware binarys i think ; )

The explanation how to extract "the right" FW from an MLNX OS image is also not my problem.
My problem rather is that the only MLNX OS version that i could find, is by HPE and only contains two FWs and not the 96 that the guide claims, and i assume those two are for the two 36Port versions HPE offers.
Could and should probably confirm that though.

But you are right, i should rather ask:

Did anyone find working FW for the 6018 and where can i get it, or additionally, where can i get the mlnxOS version that contains all of them?
 

Necrotyr

Active Member
Jun 25, 2017
206
52
28
Denmark
You have to extract it from the image you download from HP by using the hex editing step in the guide

See step 6 in the guide
 
Last edited:

Necrotyr

Active Member
Jun 25, 2017
206
52
28
Denmark
There are 3 SX6018 images in the MFA file, numbers 79,80 and 81.

Code:
MT_1240212020
MSX6018F-xxxS_Ax
SwitchX-2 based FDR InfiniBand Switch; 18 QSFP; Managed
Code:
MT_1240213020
MSX6018F-xxxS_WT_Ax
SwitchX-2 based InfiniBand to Ethernet 1U gateway; 18 QSFP+ ports; PPC460; RoHS6
Code:
MT_1240112029
MSX6018T-xxxS_Ax
SwitchX-2 based FDR-10 InfiniBand Switch; 18 QSFP; Managed
I don't know which of them you'd want, but probably MT_1240212020 or MT_1240213020.
 

RageBone

Active Member
Jul 11, 2017
617
159
43
@Necrotyr indeed. It seems as if i have looked at the wrong mfa by accident.
Extracted the one contained in the HPE 3.6.8010 image and then worked with a different one, named ...9_4_2000-6707-67-B21.mfa which is just 421KB large and seems to not be the one meant by the guide. No clue how it even got there and how i could miss the freshly extracted ...9_4_5070-FIT.mfa that is as described and expected by the guide. Also, rather large with 1.24MB.

fffffff thanks for the quick replies.
 

RageBone

Active Member
Jul 11, 2017
617
159
43
Made it through everything and i can now login with admin admin, though i'm also getting the "fatal error" that @Quantum_Mechanic also seems to have.

I used the HP mlnxOS 3.6.8010, might that be the issue here ?
 

Granulado

New Member
Sep 12, 2019
2
0
1
I just wanted to thank everybody in this thread, and especially @mpogr, @Hindsight, and @JSLEnterprises, for all the information about this. I just now finished up a successful conversion of an EMC SX6012. I do have one remaining question, although it is more out of curiosity than anything else:

When I was finished with the conversion, I had the mini-image in "Partition A" (a.k.a. mtdblock6) and the new MLNX-OS image in "Partition B" (a.k.a. mtdblock7), and the U-boot loader was defaulting to Partition A. I looked around in the U-boot environment variables and some of its documentation, but I did not see how to change the default to Partition B. In the end, I used the MLNX-OS web GUI to make the change, but I was wondering if anyone knew the under-the-hood way to do this.

Thanks again!
 
Last edited:

Granulado

New Member
Sep 12, 2019
2
0
1
Made it through everything and i can now login with admin admin, though i'm also getting the "fatal error" that @Quantum_Mechanic also seems to have.

I used the HP mlnxOS 3.6.8010, might that be the issue here ?
I had the "fatal error" problem too, and the issue for me turned out to be that the 'chad' and 'hwd' binaries were not set to permission mode 755.

This seemed strange to me, since I was _sure_ I had chmod'd them to 755 during the procedure. But when I booted back into the mini-image and checked, neither of them were set to +x. I chmod 755'd them again and rebooted into the MLNX-OS image, and everything worked fine after that.

I was using 3.6.1002 though, as the rest of the instructions and the files provided by @JSLEnterprises seemed to require (or at least assume) the use of that version.
 

Terry Wallace

PsyOps SysOp
Aug 13, 2018
197
118
43
Central Time Zone
Anyone know how to get to the uboot shell on a sx6036 that already has MellanoxOS on it ?
I can get to uboot Ctrl-b but shell option requires password. Not the usual admin/admin from web GUI.
 

JSLEnterprises

New Member
May 4, 2018
20
22
3
41
I was using 3.6.1002 though, as the rest of the instructions and the files provided by @JSLEnterprises seemed to require the use of that version.
That the is the version MPOGR explicitly stated, as the files he modified in assembly were for that specific version, and those files do change with each release. I have access to all releases for these sx switches through work, including the last one released Feb of this year (3.6.8012), and have confirmed this.
 

RageBone

Active Member
Jul 11, 2017
617
159
43
@JSLEnterprises thank you very much!

Gona try the 3.6.1002 then. Its luckily just a few of the steps i have to repeat then.
Did he ever describe how or Why he modded those binarys ?
 

Raice

Member
Jul 12, 2017
64
11
8
41
Guangzhou, China
Does anybody know safe longterm temperatures for SX6012? I have replaced all the fans with Noctua A4x20
My temperatures are:
Code:
Module    Component    Temperature [°C]    Status
MGMT    BOARD_MONITOR    40.00    OK
MGMT    CPU_BOARD_MONITOR    43.00    OK
MGMT    CPU_BOARD_MONITOR2    73.00    OK
MGMT    QSFP_TEMP1    40.50    OK
MGMT    QSFP_TEMP2    45.50    OK
MGMT    QSFP_TEMP3    44.00    OK
MGMT    SX    61.00    OK
Is this ok?
 

BeTeP

Well-Known Member
Mar 23, 2019
653
429
63
For no other reason than having too much time on my hands I finally decided to get with the program and upgrade my home network to 40GbE.
So I picked up one of those cheap EMC MSX6012 IB only switches and tried to convert it to VPI.

The conversion guide (many thanks to everyone who contributed) was pretty straight forward for the most part. The only part which took me some time to wrap my head around was the licensing issue. I am guessing that it was intentional.

I have already upgraded the MT51000 ASIC firmware to the latest version I found (9.4.5080) and my last question is whether there is any benefit of upgrading the management board software past 3.6.1002?
 

Necrotyr

Active Member
Jun 25, 2017
206
52
28
Denmark
Little sad that ACL can't be applied to VLAN interfaces until firmware 3.8.5000, can't really use it for L3 without ACLs...
 

BeTeP

Well-Known Member
Mar 23, 2019
653
429
63
3.6.8010 was the latest build of MLNX-OS that I was able to find for PPC460EX (management CPU in SX6012). And there is a thread on Mellanox community forums where a Mellanox employee claims that 3.6.8012 is the last official build for the platform. So 3.8.x does not seem like an option anyway.

Well for now I am going to stay on 3.6.1002.
 

Necrotyr

Active Member
Jun 25, 2017
206
52
28
Denmark
Si I just did im2geek4you's fan mod, set to 5v, but the fan speed actually went up ~600 rpm.

On boot the noise is noticable less, but during ordinary operation, the switch now has an annoying pitch, so I'm probably going to remove the mod again.
 

BeTeP

Well-Known Member
Mar 23, 2019
653
429
63
Does anyone know if SX6036 uses the same management board as SX6012?

I liked my SX6012 so much that I would like to upgrade it to SX6036 and my idea is to buy a cheap SX6025 and put the PPC460EX board from the SX6012 in there.
 

Freebsd1976

Active Member
Feb 23, 2018
387
73
28
Little sad that ACL can't be applied to VLAN interfaces until firmware 3.8.5000, can't really use it for L3 without ACLs...
today read release note 3.6.5000 add ACLS , there is no 3.8.5000 ,newest is 3.8.1002 . Anyway before we can get binary patch on newer 3.6.x , we can not upgrade , so still no ACLS


Release 3.6.5000
General Added GA level support for Signal Degradation Monitoring
General Configured DH default key size to 2048
ACLs Added support for UDK
See commands under “Access Control List” section in the User Manual
ACLs Added support for TCP-UDP
See commands under “Access Control List” section in the User Manual
ACLs Added support for ACL remarks
See command “remark” in the User Manual
Table 5 - Ethernet Changes and New Features
Category Description
Changes and New Features
13
Mellanox Technologies Confidential
ACLs Added support for ACL shared counters
See commands “shared-counter” and “clear shared-counter” in the User Manual
ACLs Added support for IPv6
See commands under “Access Control List” section in the User Manual
ACLs Added support for interface VLAN binding point
See command “bind-point rif” in the User Manual
ACLs Added support for L4 port range
ACLs Added support for logging
See the command “access-list log” in the User Manual
ACLs Added support for TCP flags
See the commands “deny/permit (IPv4 TCP ACL rule)” and “deny/permit (IPv6 TCP
ACL rule)” in the User Manual
HLL Added support for HLL
See section “Head-of-Queue Lifetime Limit” in the User Manual
IGMP Querier Added GA support for IGMP Querier
JSON Added support for additional JSON commands
See Appendix “Show Commands Supported by JSON API”
PIM Added GA level support for PIM SSM
Shared Buffers Added support for user mode configuration
Shared Buffers Enhanced and simplified advanced mode configuration
Telemetry Added support for Ethernet thresholds telemetry
UDK Added support for UDK
See section “User Defined Keys” in the User Manual
WebUI Added new IP Interfaces page
See IP Route>IP Interfaces in the WebUI
WebUI Added support for JSON batch commands
ZTP Added support for DHCP based ZTP
See section “Zero-touch Provisioning” in the User Manual