Beware of EMC switches sold as Mellanox SX6XXX on eBay

Discussion in 'Networking' started by mpogr, Aug 2, 2016.

  1. RageBone

    RageBone Active Member

    Joined:
    Jul 11, 2017
    Messages:
    250
    Likes Received:
    63
    What? Sorry but i don't understand how or where you mean.
    I have the PDF and docx versions of the guide, the later from the JS.. archive, in which there is a prepared binary for the sx6012 but not the 6018 and i haven't grasped that those would be compatible.
    The docx or pdf itself don't contain any firmware binarys i think ; )

    The explanation how to extract "the right" FW from an MLNX OS image is also not my problem.
    My problem rather is that the only MLNX OS version that i could find, is by HPE and only contains two FWs and not the 96 that the guide claims, and i assume those two are for the two 36Port versions HPE offers.
    Could and should probably confirm that though.

    But you are right, i should rather ask:

    Did anyone find working FW for the 6018 and where can i get it, or additionally, where can i get the mlnxOS version that contains all of them?
     
    #421
  2. Necrotyr

    Necrotyr Member

    Joined:
    Jun 25, 2017
    Messages:
    109
    Likes Received:
    15
    You have to extract it from the image you download from HP by using the hex editing step in the guide

    See step 6 in the guide
     
    #422
    Last edited: Sep 5, 2019
  3. Necrotyr

    Necrotyr Member

    Joined:
    Jun 25, 2017
    Messages:
    109
    Likes Received:
    15
    There are 3 SX6018 images in the MFA file, numbers 79,80 and 81.

    Code:
    MT_1240212020
    MSX6018F-xxxS_Ax
    SwitchX-2 based FDR InfiniBand Switch; 18 QSFP; Managed
    Code:
    MT_1240213020
    MSX6018F-xxxS_WT_Ax
    SwitchX-2 based InfiniBand to Ethernet 1U gateway; 18 QSFP+ ports; PPC460; RoHS6
    Code:
    MT_1240112029
    MSX6018T-xxxS_Ax
    SwitchX-2 based FDR-10 InfiniBand Switch; 18 QSFP; Managed
    I don't know which of them you'd want, but probably MT_1240212020 or MT_1240213020.
     
    #423
  4. RageBone

    RageBone Active Member

    Joined:
    Jul 11, 2017
    Messages:
    250
    Likes Received:
    63
    @Necrotyr indeed. It seems as if i have looked at the wrong mfa by accident.
    Extracted the one contained in the HPE 3.6.8010 image and then worked with a different one, named ...9_4_2000-6707-67-B21.mfa which is just 421KB large and seems to not be the one meant by the guide. No clue how it even got there and how i could miss the freshly extracted ...9_4_5070-FIT.mfa that is as described and expected by the guide. Also, rather large with 1.24MB.

    fffffff thanks for the quick replies.
     
    #424
  5. RageBone

    RageBone Active Member

    Joined:
    Jul 11, 2017
    Messages:
    250
    Likes Received:
    63
    Made it through everything and i can now login with admin admin, though i'm also getting the "fatal error" that @Quantum_Mechanic also seems to have.

    I used the HP mlnxOS 3.6.8010, might that be the issue here ?
     
    #425
  6. Granulado

    Granulado New Member

    Joined:
    Sep 12, 2019
    Messages:
    2
    Likes Received:
    0
    I just wanted to thank everybody in this thread, and especially @mpogr, @Hindsight, and @JSLEnterprises, for all the information about this. I just now finished up a successful conversion of an EMC SX6012. I do have one remaining question, although it is more out of curiosity than anything else:

    When I was finished with the conversion, I had the mini-image in "Partition A" (a.k.a. mtdblock6) and the new MLNX-OS image in "Partition B" (a.k.a. mtdblock7), and the U-boot loader was defaulting to Partition A. I looked around in the U-boot environment variables and some of its documentation, but I did not see how to change the default to Partition B. In the end, I used the MLNX-OS web GUI to make the change, but I was wondering if anyone knew the under-the-hood way to do this.

    Thanks again!
     
    #426
    Last edited: Sep 12, 2019
  7. Freebsd1976

    Freebsd1976 Member

    Joined:
    Feb 23, 2018
    Messages:
    97
    Likes Received:
    7
    only 3.6.1002 , no more , no less
     
    #427
  8. Granulado

    Granulado New Member

    Joined:
    Sep 12, 2019
    Messages:
    2
    Likes Received:
    0
    I had the "fatal error" problem too, and the issue for me turned out to be that the 'chad' and 'hwd' binaries were not set to permission mode 755.

    This seemed strange to me, since I was _sure_ I had chmod'd them to 755 during the procedure. But when I booted back into the mini-image and checked, neither of them were set to +x. I chmod 755'd them again and rebooted into the MLNX-OS image, and everything worked fine after that.

    I was using 3.6.1002 though, as the rest of the instructions and the files provided by @JSLEnterprises seemed to require (or at least assume) the use of that version.
     
    #428
  9. Terry Wallace

    Terry Wallace PsyOps SysOp

    Joined:
    Aug 13, 2018
    Messages:
    84
    Likes Received:
    35
    Anyone know how to get to the uboot shell on a sx6036 that already has MellanoxOS on it ?
    I can get to uboot Ctrl-b but shell option requires password. Not the usual admin/admin from web GUI.
     
    #429
  10. joddlarn

    joddlarn Member

    Joined:
    Jul 14, 2018
    Messages:
    30
    Likes Received:
    7
    #430
    metag and Terry Wallace like this.
  11. Terry Wallace

    Terry Wallace PsyOps SysOp

    Joined:
    Aug 13, 2018
    Messages:
    84
    Likes Received:
    35
    I swear I had read this whole thread at least twice. Facepalm.
    Thanks
     
    #431
  12. JSLEnterprises

    JSLEnterprises New Member

    Joined:
    May 4, 2018
    Messages:
    20
    Likes Received:
    21
    That the is the version MPOGR explicitly stated, as the files he modified in assembly were for that specific version, and those files do change with each release. I have access to all releases for these sx switches through work, including the last one released Feb of this year (3.6.8012), and have confirmed this.
     
    #432
  13. RageBone

    RageBone Active Member

    Joined:
    Jul 11, 2017
    Messages:
    250
    Likes Received:
    63
    @JSLEnterprises thank you very much!

    Gona try the 3.6.1002 then. Its luckily just a few of the steps i have to repeat then.
    Did he ever describe how or Why he modded those binarys ?
     
    #433
  14. Raice

    Raice Member

    Joined:
    Jul 12, 2017
    Messages:
    40
    Likes Received:
    5
    Does anybody know safe longterm temperatures for SX6012? I have replaced all the fans with Noctua A4x20
    My temperatures are:
    Code:
    Module    Component    Temperature [°C]    Status
    MGMT    BOARD_MONITOR    40.00    OK
    MGMT    CPU_BOARD_MONITOR    43.00    OK
    MGMT    CPU_BOARD_MONITOR2    73.00    OK
    MGMT    QSFP_TEMP1    40.50    OK
    MGMT    QSFP_TEMP2    45.50    OK
    MGMT    QSFP_TEMP3    44.00    OK
    MGMT    SX    61.00    OK
    
    Is this ok?
     
    #434
  15. BeTeP

    BeTeP Active Member

    Joined:
    Mar 23, 2019
    Messages:
    228
    Likes Received:
    84
    For no other reason than having too much time on my hands I finally decided to get with the program and upgrade my home network to 40GbE.
    So I picked up one of those cheap EMC MSX6012 IB only switches and tried to convert it to VPI.

    The conversion guide (many thanks to everyone who contributed) was pretty straight forward for the most part. The only part which took me some time to wrap my head around was the licensing issue. I am guessing that it was intentional.

    I have already upgraded the MT51000 ASIC firmware to the latest version I found (9.4.5080) and my last question is whether there is any benefit of upgrading the management board software past 3.6.1002?
     
    #435
  16. Necrotyr

    Necrotyr Member

    Joined:
    Jun 25, 2017
    Messages:
    109
    Likes Received:
    15
    Little sad that ACL can't be applied to VLAN interfaces until firmware 3.8.5000, can't really use it for L3 without ACLs...
     
    #436
  17. BeTeP

    BeTeP Active Member

    Joined:
    Mar 23, 2019
    Messages:
    228
    Likes Received:
    84
    3.6.8010 was the latest build of MLNX-OS that I was able to find for PPC460EX (management CPU in SX6012). And there is a thread on Mellanox community forums where a Mellanox employee claims that 3.6.8012 is the last official build for the platform. So 3.8.x does not seem like an option anyway.

    Well for now I am going to stay on 3.6.1002.
     
    #437
  18. Necrotyr

    Necrotyr Member

    Joined:
    Jun 25, 2017
    Messages:
    109
    Likes Received:
    15
    Si I just did im2geek4you's fan mod, set to 5v, but the fan speed actually went up ~600 rpm.

    On boot the noise is noticable less, but during ordinary operation, the switch now has an annoying pitch, so I'm probably going to remove the mod again.
     
    #438
  19. BeTeP

    BeTeP Active Member

    Joined:
    Mar 23, 2019
    Messages:
    228
    Likes Received:
    84
    Does anyone know if SX6036 uses the same management board as SX6012?

    I liked my SX6012 so much that I would like to upgrade it to SX6036 and my idea is to buy a cheap SX6025 and put the PPC460EX board from the SX6012 in there.
     
    #439
  20. Freebsd1976

    Freebsd1976 Member

    Joined:
    Feb 23, 2018
    Messages:
    97
    Likes Received:
    7
    today read release note 3.6.5000 add ACLS , there is no 3.8.5000 ,newest is 3.8.1002 . Anyway before we can get binary patch on newer 3.6.x , we can not upgrade , so still no ACLS


    Release 3.6.5000
    General Added GA level support for Signal Degradation Monitoring
    General Configured DH default key size to 2048
    ACLs Added support for UDK
    See commands under “Access Control List” section in the User Manual
    ACLs Added support for TCP-UDP
    See commands under “Access Control List” section in the User Manual
    ACLs Added support for ACL remarks
    See command “remark” in the User Manual
    Table 5 - Ethernet Changes and New Features
    Category Description
    Changes and New Features
    13
    Mellanox Technologies Confidential
    ACLs Added support for ACL shared counters
    See commands “shared-counter” and “clear shared-counter” in the User Manual
    ACLs Added support for IPv6
    See commands under “Access Control List” section in the User Manual
    ACLs Added support for interface VLAN binding point
    See command “bind-point rif” in the User Manual
    ACLs Added support for L4 port range
    ACLs Added support for logging
    See the command “access-list log” in the User Manual
    ACLs Added support for TCP flags
    See the commands “deny/permit (IPv4 TCP ACL rule)” and “deny/permit (IPv6 TCP
    ACL rule)” in the User Manual
    HLL Added support for HLL
    See section “Head-of-Queue Lifetime Limit” in the User Manual
    IGMP Querier Added GA support for IGMP Querier
    JSON Added support for additional JSON commands
    See Appendix “Show Commands Supported by JSON API”
    PIM Added GA level support for PIM SSM
    Shared Buffers Added support for user mode configuration
    Shared Buffers Enhanced and simplified advanced mode configuration
    Telemetry Added support for Ethernet thresholds telemetry
    UDK Added support for UDK
    See section “User Defined Keys” in the User Manual
    WebUI Added new IP Interfaces page
    See IP Route>IP Interfaces in the WebUI
    WebUI Added support for JSON batch commands
    ZTP Added support for DHCP based ZTP
    See section “Zero-touch Provisioning” in the User Manual
     
    #440
Similar Threads: Beware switches
Forum Title Date
Networking Beware - Emulex 10GbE Virtual Fabric Adapter II x8 PCI-E slot Jun 22, 2014
Networking Multiple Subnets on Multiple Switches, No VLAN Sep 22, 2019
Networking [EU] Looking for cheap SFP+ switches and QSFP+ backbone switch Jun 15, 2019
Networking Mellanox MIS5035 Switches - 40gb --> (4) 10gb May 13, 2019
Networking DELL started to vendor-lock switches again Apr 1, 2019

Share This Page