Beware of EMC switches sold as Mellanox SX6XXX on eBay

efschu2 · Apr 11, 2019

In the guide I have gotten there is no ibd mentioned to be replaced?

Labs · Apr 11, 2019

In uboot I found the command "nand". This command which seems to read from nand into memory. Since nand is 1GB and RAM is 2GB it should be possible to load the full nand flash into RAM and after upload it to TFTP or USB.

The format of the command is:
nand read <address> <nand offset> <size>

For bad block management we should use:
nand[.jffs2] read <address> <nand offset> <size>

Problem is that I couldn't find any tftp command only tftpboot.
There is also usb command but I didn't check the parameters for this one...
According to some documentation for uboot the command usb write should exist.

This uboot version is from 2009 so some commands might have been added in later versions.

Another command is md - memory display which can be used to display the RAM contents after nand read but it will probably take ages to display it and copy/paste from serial to some file.

Use printenv to check the environment variables and bdinfo also shows more info for mem/flash/sram start/size.

EDIT: There is another interesting command in uboot called imls - list all images found in flash. Maybe it can show the list of images and we can dump one by one but still not clear how to upload them via tftp or usb.

Mitron · Apr 11, 2019

mpogr said:
Just wanted to give heads up here that there is currently a seller from Ireland selling SX6012 switches on eBay (Mellanox SX6012 12-port Non-blocking Managed 56Gb/s InfiniBand/VPI SDN Switch) for a really low price. These are not Mellanox genuine switches and they don't run MLNX-OS! Beware and avoid disappointment!

If you don't mind using them as unmanaged switches (Infiniband only), you can flash them with the SX6005 firmware and it'll work fine, including FDR.

I'm currently hacking my way to still make MLNX-OS work on this switch, but it's a major hassle, involving heavy distribution modification, and still haven't been able to make it work fully. Please, PM me if you want to know more details and/or contribute to this work.

I have 6 of the SX6012 switches and access to the images and firmware on the Mellanox site, but of course no documemtation to convert the EMC OS to the Mellanox OS, nor do they have docs to reset them to factory defaults and clear previous customer info. I was able to bring one back to life that wouldnt boot, but it still has the EMC OS. Any help to convert these to the Mellanox OS would be appreciated.

Quantum_Mechanic · Apr 11, 2019

is there something i need to do between steps 5 and 6?
im not getting an IB link with my linux box.. If I plug my linux box into a different "dumb" switch I get a link.

Freebsd1976 · Apr 11, 2019

Quantum_Mechanic said:
is there something i need to do between steps 5 and 6?
im not getting an IB link with my linux box.. If I plug my linux box into a different "dumb" switch I get a link.

you need boot to emcos ， run imls find emcos then run ， everything should ok

Quantum_Mechanic · Apr 12, 2019

tnx Freebsd!..

Necrotyr · Apr 12, 2019

In step 2 of the guide you are supposed to symlink init to busybox with this command:

Code:

ln –s busybox ../sbin/init

but this results in a broken symlink, wouldn't the command be

Code:

ln –s ../bin/busybox ../sbin/init

instead or is a broken symlink okay?

First two are broken last is okay.

kvm_guy · Apr 18, 2019

Have I understood this correct? For the original Mellanox SX6036 an ethernet upgrade license is needed and a community member is selling them? So he is selling illegal licenses? ... weired

Rand__ · Apr 18, 2019

Its a new member with no posts yet, so not somebody from this thread

zxv · Apr 18, 2019

Does iSER work well on these switches?

fohdeesha · Apr 18, 2019

kvm_guy said:
Have I understood this correct? For the original Mellanox SX6036 an ethernet upgrade license is needed and a community member is selling them? So he is selling illegal licenses? ... weired

where in the world did you see that?

fohdeesha · Apr 18, 2019

wait, these are all PowerPC based? I might have to pick one up, I have quite a bit of hardware for um..."debugging" PPC

arglebargle · Apr 18, 2019

fohdeesha said:
wait, these are all PowerPC based? I might have to pick one up, I have quite a bit of hardware for um..."debugging" PPC

~~I think the SX60xx are some all in one ARM + NIC SoC, I'm not 100% on that though.~~

Quantum_Mechanic · Apr 18, 2019

I've tried 3 devices so far. two emc sx6018 and one emc sx6012. the mod doesn't work. Either the instructions are incorrect or Ive got bad files. I studied the docs and every page of this thread over and over and over. I even wrote my own docs. first boot I can never log in with admin/admin so i break the links and swap out passwd and shadow, reboot them im in. I try starting cli from bash and it appears to be "configuring modules" for 15 minutes then crashes "fatal internal error occured"

Code:

md5
326a90195938c19523888bbf6fab19bc    chad
37c53191f8a1749ebdb42463d9454d88    fdt_uni
e6df18fddae5f10412eabd2755f81854    hwd
55ca2838e036872830bd35987e1cb1d3    ibd
f5faeb61a5a11b7b67f14735fe81406d    image_layout.sh
183345ad0d9b08d734fbab14b0228153    image.tgz
f9f87a0e10b9596786eaa75acd80729b    vmlinuz-uni

Code:

oot@n4:~# flint -i sx6012F_wt.bin q
Image type:            FS2
FW Version:            9.3.8170
FW Release Date:       6.6.2016
Device ID:             51000
Description:           Node             Sys image
GUIDs:                 0000000000000000 0000000000000000
Description:           Base             Switch
MACs:                      000000000000     000000000000
VSD:                   n/a
PSID:                  MT_1270111029
root@n4:~# flint --allow_psid_change -d lid-0x5 -i ./sx6012F_wt.bin b

    Current FW version on flash:  9.9.1260
    New FW version:               9.3.8170

    Note: The new FW version is older than the current FW version on flash.

 Do you want to continue ? (y/n) [n] : y


    You are about to replace current PSID on flash - "EMC1270110020" with a different PSID - "MT_1270111029".
    Note: It is highly recommended not to change the PSID.

 Do you want to continue ? (y/n) [n] : y
Burning FS2 FW image without signatures - OK
Restoring signature                     - OK
root@n4:~#

Code:

localhost login: admin
Last login: Sun Feb 11 20:17:26 on ttyS0
Feb 11 20:35:04 localhost mgmtd[4089]: [mgmtd.ERR]: md_db_process_trusted_auth_info(), mdm_db.c:9209, build 1: Required condition was not met
Feb 11 20:35:04 localhost cli[4621]: [cli.ERR]: user admin: cli_cluster_cmds_init(), cli_cluster_cmds.c:205, build 1: failed getting /cluster/config/enable
    System is initializing!
This may take a few minutes


    Modules are being configured
Feb 11 20:35:11 localhost xinetd[4622]: open( /var/opt/tms/output/xinetd.conf ) failed: No such file or directory (errno = 2)
Feb 11 20:35:11 localhost xinetd[4622]: 4622 {init_services} couldn't get configuration. Exiting...
Feb 11 20:35:15 localhost chad[4630]: [chad.ERR]: bn_binding_get_uint16(), bnode_types.c:3034, build 1: NULL binding passed
Feb 11 20:35:15 localhost chad[4630]: [chad.ERR]: s2m_init(), chad_2node.c:5453, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:35:15 localhost chad[4630]: [chad.ERR]: chad_init_internal(), chad.c:2761, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:35:15 localhost chad[4630]: [chad.ERR]: main(), chad.c:2917, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:35:19 localhost httpd[4638]: httpd: Could not open configuration file /var/opt/tms/output/httpd.conf: No such file or directory
Feb 11 20:36:33 localhost xinetd[4654]: open( /var/opt/tms/output/xinetd.conf ) failed: No such file or directory (errno = 2)
Feb 11 20:36:33 localhost xinetd[4654]: 4654 {init_services} couldn't get configuration. Exiting...
Feb 11 20:36:37 localhost chad[4662]: [chad.ERR]: bn_binding_get_uint16(), bnode_types.c:3034, build 1: NULL binding passed
Feb 11 20:36:37 localhost chad[4662]: [chad.ERR]: s2m_init(), chad_2node.c:5453, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:36:37 localhost chad[4662]: [chad.ERR]: chad_init_internal(), chad.c:2761, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:36:37 localhost chad[4662]: [chad.ERR]: main(), chad.c:2917, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:36:41 localhost httpd[4670]: httpd: Could not open configuration file /var/opt/tms/output/httpd.conf: No such file or directory
Feb 11 20:40:36 localhost xinetd[4686]: open( /var/opt/tms/output/xinetd.conf ) failed: No such file or directory (errno = 2)
Feb 11 20:40:36 localhost xinetd[4686]: 4686 {init_services} couldn't get configuration. Exiting...
Feb 11 20:40:40 localhost chad[4694]: [chad.ERR]: bn_binding_get_uint16(), bnode_types.c:3034, build 1: NULL binding passed
Feb 11 20:40:40 localhost chad[4694]: [chad.ERR]: s2m_init(), chad_2node.c:5453, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:40:40 localhost chad[4694]: [chad.ERR]: chad_init_internal(), chad.c:2761, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:40:40 localhost chad[4694]: [chad.ERR]: main(), chad.c:2917, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:40:45 localhost httpd[4702]: httpd: Could not open configuration file /var/opt/tms/output/httpd.conf: No such file or directory
Feb 11 20:46:33 localhost snmpd[4461]: TID 1208140448: [snmpd.ERR]: sn_mib_get_caching_interval(), sn_mib_mellanox.c:173, build 1: Unexpected NULL
Feb 11 20:46:33 localhost snmpd[4461]: TID 1208140448: [snmpd.ERR]: sn_mib_entity_init(), sn_mib_entity.c:76, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_sm_login_banner(), cli_sm_cmds.c:21490, build 1: Error code 14022 (type conversion failed, or bad data type provided) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_sm_init_sm_ha(), cli_sm_cmds.c:987, build 1: Error code 14022 (type conversion failed, or bad data type provided) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_sm_cmds_init(), cli_sm_cmds.c:11627, build 1: Error code 14022 (type conversion failed, or bad data type provided) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: util_get_uint8(), libutils.c:57, build 1: Unexpected NULL
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: lib_ui_api_get_system_profile(), lib_ui_api_utils.c:667, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_ports_cmds_init(), cli_ports_cmds.c:169, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: util_get_uint8(), libutils.c:57, build 1: Unexpected NULL
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_chassis_cmds_init(), cli_chassis_cmds.c:6457, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_ip_cmds_init(), cli_ip_cmds.c:344, build 1: Unexpected NULL
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: update_g_ip_proxy_arp_active(), cli_hide_cmds.c:1024, build 1: Unexpected NULL
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: chc_get_ip_proxy_arp_active(), cli_hide_cmds.c:1058, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: lib_ui_api_check_ip_l3_config(), lib_ui_api.c:6775, build 1: Unexpected NULL
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: chc_get_ip_l3_active(), cli_hide_cmds.c:3878, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost mgmtd[4089]: [mgmtd.ERR]: md_openflow_openflow_enabled(), md_openflowd.c:216, build 1: Required condition was not met
Feb 11 20:47:06 localhost mgmtd[4089]: [mgmtd.ERR]: md_openflow_handle_state_protocol(), md_openflowd.c:236, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:06 localhost mgmtd[4089]: [mgmtd.ERR]: mdb_query(), mdb_db.c:3060, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:06 localhost mgmtd[4089]: [mgmtd.ERR]: md_commit_handle_query_request(), md_commit.c:4098, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:06 localhost mgmtd[4089]: [mgmtd.ERR]: md_commit_handle_query_session_request(), md_commit.c:4290, build 1: Error code 14002 (assertion failed) returned
% Internal error, code 1006 (see logs for details)
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: update_g_router_state(), cli_hide_cmds.c:4479, build 1: Unexpected NULL
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: chc_get_router_state(), cli_hide_cmds.c:4466, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: chc_license_init(), cli_hide_cmds.c:799, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_hide_cmds_init(), cli_hide_cmds.c:510, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:06 localhost cli[4621]: [cli.ERR]: user admin: cli_init_modules(), cli_modules.c:455, build 1: Error code 14001 (unexpected NULL) returned
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: cli_register_help_handler(), cli_registration.c:1764, build 1: Help description can only be specified for literal command words; use cc_help_exp instead
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: Cmd reg: error registering command "fae interface *"
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: cli_register_command(), cli_registration.c:377, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: cli_sanity_check_node_internal(), cli_registration.c:1155, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: tree_foreach_internal(), tree.c:561, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: tree_foreach_internal(), tree.c:595, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: tree_foreach_internal(), tree.c:595, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: cli_register_help_handler(), cli_registration.c:1764, build 1: Help description can only be specified for literal command words; use cc_help_exp instead
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: Cmd reg: error registering command "fae no interface *"
Feb 11 20:47:07 localhost cli[4621]: [cli.ERR]: user admin: cli_register_command(), cli_registration.c:377, build 1: Error code 14002 (assertion failed) returned
Feb 11 20:47:0Feb 11FeFeb 11 20:47:07 localhost last message repeated 3 times
Feb 11 20:47:07 localhost cli[4621]: [cli.EFeFeFeb 11FFeb 11 Feb 11 20:47:07% A fatal internal error occurred

lambda · Apr 18, 2019

fohdeesha said:
wait, these are all PowerPC based? I might have to pick one up, I have quite a bit of hardware for um..."debugging" PPC

SX6000 series is based on the SwitchX ASIC. The SwitchX Development Kit has an integrated PPC CPU.

http://www.mellanox.com/page/products_dyn?product_family=126&mtag=switchx_dvk

fohdeesha · Apr 18, 2019

arglebargle said:
I think the SX60xx are some all in one ARM + NIC SoC, I'm not 100% on that though.

I keep seeing filenames like image-PPC_M460EX-ppc-m460ex-20160609-202426.tgz being passed around, are these directly for the SX series or from another model that's just being used to grab some files out of? Because that's definitely PPC firmware

fohdeesha · Apr 18, 2019

lambda said:
SX6000 series is based on the SwitchX ASIC. The SwitchX Development Kit has an integrated PPC CPU.

http://www.mellanox.com/page/products_dyn?product_family=126&mtag=switchx_dvk

perfect, that's what I thought. Looks like the 460 series: https://datasheet.octopart.com/PPC460EX-NUB800T-AMCC-datasheet-11553412.pdf

Code:

* Chasis Type        : DINGO
* Number of Ports    : 12
* U-Boot Revision    :U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82-EMC ppc (Feb 27 2013 - 12:13:42)

arglebargle · Apr 18, 2019

Can someone point me to firmware downloads if they're available, or if not send me a pm with a private link? I'd like to start hacking on my SX6012.

Danke!

lambda · Apr 18, 2019

fohdeesha said:
perfect, that's what I thought. Looks like the 460 series: https://datasheet.octopart.com/PPC460EX-NUB800T-AMCC-datasheet-11553412.pdf

Code:

* Chasis Type : DINGO * Number of Ports : 12 * U-Boot Revision :U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82-EMC ppc (Feb 27 2013 - 12:13:42)

There are 27 occurrences of the string "460EX" in this document:
https://www.mellanox.com/related-docs/prod_management_software/MLNX-OS_ETH_v3_6_3508_UM.pdf

Rand__ · Apr 19, 2019

efschu2 said:
In the guide I have gotten there is no ibd mentioned to be replaced?

ibd was a later addition that might not have been updated in the doc

fohdeesha said:
where in the world did you see that?

My post here: https://forums.servethehome.com/index.php?threads/mellanox-sx6036-36ports-qsfp-€150.24158/page-2#post-225050

arglebargle said:
Can someone point me to firmware downloads if they're available, or if not send me a pm with a private link? I'd like to start hacking on my SX6012.

https://forums.servethehome.com/index.php?threads/mellanox-sx6036-36ports-qsfp-€150.24158/page-2#post-225295
That post should give you a starting point to search for a 6012 image, just make sure to get the correct level as there is no feedback on whether it works with a more recent firmware

Quantum_Mechanic said:

Code:

md5
326a90195938c19523888bbf6fab19bc    chad
37c53191f8a1749ebdb42463d9454d88    fdt_uni
e6df18fddae5f10412eabd2755f81854    hwd
55ca2838e036872830bd35987e1cb1d3    ibd
f5faeb61a5a11b7b67f14735fe81406d    image_layout.sh
183345ad0d9b08d734fbab14b0228153    image.tgz
f9f87a0e10b9596786eaa75acd80729b    vmlinuz-uni

il.sh, hwd, ibd, chad match my files, not sure where fdt_uni comes into play and the other two are generated, are they not?

Beware of EMC switches sold as Mellanox SX6XXX on eBay

Member

Member

New Member

New Member

Active Member

New Member

Active Member

New Member

Well-Known Member

The more I C, the less I see.

Kaini Industries

Kaini Industries

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

New Member

Member

Kaini Industries

Kaini Industries

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

Member

Well-Known Member