Beware of EMC switches sold as Mellanox SX6XXX on eBay

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
S

Softlution

New Member
Jul 29, 2022
16
1
3
Is this a successful conversion or i missed something? I plugged a qsfp cable on the switch but there are no lights blinking and it doesn't seem to work. I did this a long time ago and I don't remember the login credentials. Any way to fix that? Also, how to I access the web interface?

INIT: version 2.86 booting

Starting: SX_PPC_M460EX SX_3.4.0012 2014-12-15 23:27:42 ppc
Starting udev: [ OK ]
Setting clock (utc): Mon Jul 31 18:58:56 UTC 2000 [ OK ]
Setting hostname localhost: [ OK ]
Checking filesystems
Checking all file systems.
[ OK ]
Remounting root filesystem in read-write mode: [ OK ]
Mounting local filesystems: [ OK ]
Running vpart script: [ OK ]
Applying file system skeletons: base_var base_config .
Enabling /etc/fstab swaps: [ OK ]
INIT: Entering runlevel: 3
Starting system services
Starting sx_low_level_if: Starting sx_low_level_if:
Loading i2c_mux_pca954x driver - Success
Loading glue logic low level - Success
Loading watchdog - Success
Loading cpld handler - Success
Loading mellaggra module - Success
Loading sx i2c module - Success
[ OK ]
Starting openibd: IPoIB configuration for embedded system
Loading SX driver:[ OK ]
Loading HCA driver and Access Layer:[ OK ]
Setting up InfiniBand network interfaces:
Setting up service network . . .[ done ]
Reloading udev:
[ OK ]
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Running renaming interfaces
Renaming: MAC: E4:1D:2D:F3:48:44 ifindex: 2 name: mgmt0
Renaming: MAC: E4:1D:2D:F3:48:45 ifindex: 3 name: mgmt1
Checking for unexpected shutdown
Detected unexpected shutdown!
Probing for HRNG module
Starting rngd: [ OK ]
Running system image: SX_PPC_M460EX SX_3.4.0012 2014-12-15 23:27:42 ppc
Applying initial configuration: Jul 31 18:59:39 INFO LOG: Initializing SX log with STDOUT as output file.


Starting internal_startup: [ OK ]
Starting tc_ingress_policy: tc_ingress_policy system name is not obtained - use default IS5600MDC
mDNS policing rate=4000kbit burst=400k
Ingress policing enable on interface mgmt0 rate=9000kbit burst=900k
[ OK ]
Starting clean_issnvram: Deleting issnvram.txt
[ OK ]
Starting intr_hndl: Starting :
Loading int handler module - Success
[ OK ]
Starting iss-nvram-mac: [ OK ]
Starting copy_rh_files_to_vtmp: [ OK ]
Starting sx_pra: Starting proxy arp management:
Loading proxy arp management module - Success
[ OK ]
Starting udevd: Reloading udev...
[ OK ]
Starting pm: [ OK ]
Starting oops_dump_reg: Starting kernel reg dump:
Loading kernel reg dump module - Success
[ OK ]
Starting lnpuppetvar.sh: [ OK ]
Starting mst: Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
[ OK ]


Mellanox MLNX-OS Switch Management

switch-f34844 login:
 
F

Freebsd1976

Active Member
Feb 23, 2018
390
73
28
Softlution said:
Is this a successful conversion or i missed something? I plugged a qsfp cable on the switch but there are no lights blinking and it doesn't seem to work. I did this a long time ago and I don't remember the login credentials. Any way to fix that? Also, how to I access the web interface?

INIT: version 2.86 booting

Starting: SX_PPC_M460EX SX_3.4.0012 2014-12-15 23:27:42 ppc
Starting udev: [ OK ]
Setting clock (utc): Mon Jul 31 18:58:56 UTC 2000 [ OK ]
Setting hostname localhost: [ OK ]
Checking filesystems
Checking all file systems.
[ OK ]
Remounting root filesystem in read-write mode: [ OK ]
Mounting local filesystems: [ OK ]
Running vpart script: [ OK ]
Applying file system skeletons: base_var base_config .
Enabling /etc/fstab swaps: [ OK ]
INIT: Entering runlevel: 3
Starting system services
Starting sx_low_level_if: Starting sx_low_level_if:
Loading i2c_mux_pca954x driver - Success
Loading glue logic low level - Success
Loading watchdog - Success
Loading cpld handler - Success
Loading mellaggra module - Success
Loading sx i2c module - Success
[ OK ]
Starting openibd: IPoIB configuration for embedded system
Loading SX driver:[ OK ]
Loading HCA driver and Access Layer:[ OK ]
Setting up InfiniBand network interfaces:
Setting up service network . . .[ done ]
Reloading udev:
[ OK ]
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Running renaming interfaces
Renaming: MAC: E4:1D:2D:F3:48:44 ifindex: 2 name: mgmt0
Renaming: MAC: E4:1D:2D:F3:48:45 ifindex: 3 name: mgmt1
Checking for unexpected shutdown
Detected unexpected shutdown!
Probing for HRNG module
Starting rngd: [ OK ]
Running system image: SX_PPC_M460EX SX_3.4.0012 2014-12-15 23:27:42 ppc
Applying initial configuration: Jul 31 18:59:39 INFO LOG: Initializing SX log with STDOUT as output file.


Starting internal_startup: [ OK ]
Starting tc_ingress_policy: tc_ingress_policy system name is not obtained - use default IS5600MDC
mDNS policing rate=4000kbit burst=400k
Ingress policing enable on interface mgmt0 rate=9000kbit burst=900k
[ OK ]
Starting clean_issnvram: Deleting issnvram.txt
[ OK ]
Starting intr_hndl: Starting :
Loading int handler module - Success
[ OK ]
Starting iss-nvram-mac: [ OK ]
Starting copy_rh_files_to_vtmp: [ OK ]
Starting sx_pra: Starting proxy arp management:
Loading proxy arp management module - Success
[ OK ]
Starting udevd: Reloading udev...
[ OK ]
Starting pm: [ OK ]
Starting oops_dump_reg: Starting kernel reg dump:
Loading kernel reg dump module - Success
[ OK ]
Starting lnpuppetvar.sh: [ OK ]
Starting mst: Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
[ OK ]


Mellanox MLNX-OS Switch Management

switch-f34844 login:
Click to expand...
press and hold reset button for LONG LONG LONG LONG time (10-15 minutes, watch console output) , switch will go factory default.
 
S

Softlution

New Member
Jul 29, 2022
16
1
3
Freebsd1976 said:
press and hold reset button for LONG LONG LONG LONG time (10-15 minutes, watch console output) , switch will go factory default.
Click to expand...
Thanks for your reply. I tried resetting but no luck. I started from scratch but I got stack on this.

== Using image from URL: http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img

== Calling writeimage to image system
*** Partition /dev/mtdblock6 was mounted and should not be
Error: writeimage failed, exiting.
====== Ending image manufacture with error at 20240310-234448
 
Last edited:
S

Softlution

New Member
Jul 29, 2022
16
1
3
I got this. Could you please help me understand the situation and how it can be fixed?

# /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img
====== Starting manufacture at 20240310-233634
====== Called as: /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img

==================================================
Manufacture script starting
==================================================

== Using model: ppc
== Using kernel type: uni
== Using layout: MFL1
== Using partition name-size list:
== Using device list: /dev/mtd
== Using interface list: mgmt0 mgmt1
== Using interface naming: ifindex-sorted
== Smartd disabled
== Cluster enable: no
== Cluster ID: (none)
== Cluster description: (none)
== Cluster interface: (none)
== Cluster master virtual IP address: 0.0.0.0
== Cluster master virtual IP masklen: 0
== Cluster shared secret: (none)
== Cluster expected number of nodes: 0
- Assigning specified interface names in ifindex-sorted order
-- Mapping MAC: E4:1D:2D:F3:47:1C from: eth0 to: mgmt0
-- Mapping MAC: E4:1D:2D:F3:47:1D from: eth1 to: mgmt1
== Using image from URL: http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img

== Calling writeimage to image system
zcat: write: No space left on device
zcat: crc error
zcat: Incorrect length
*** Imaging interrupted
*** Manufacture interrupted
====== Ending image manufacture by trap at 20240310-234011
# /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img
====== Starting manufacture at 20240310-234445
====== Called as: /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img

==================================================
Manufacture script starting
==================================================

== Using model: ppc
== Using kernel type: uni
== Using layout: MFL1
== Using partition name-size list:
== Using device list: /dev/mtd
== Using interface list: mgmt0 mgmt1
== Using interface naming: ifindex-sorted
== Smartd disabled
== Cluster enable: no
== Cluster ID: (none)
== Cluster description: (none)
== Cluster interface: (none)
== Cluster master virtual IP address: 0.0.0.0
== Cluster master virtual IP masklen: 0
== Cluster shared secret: (none)
== Cluster expected number of nodes: 0
- Assigning specified interface names in ifindex-sorted order
-- Mapping MAC: E4:1D:2D:F3:47:1C from: eth0 to: mgmt0
-- Mapping MAC: E4:1D:2D:F3:47:1D from: eth1 to: mgmt1
== Using image from URL: http://192.168.7.77/web_share/image-PPC_M460EX-3.4.2008.img

== Calling writeimage to image system
*** Partition /dev/mtdblock6 was mounted and should not be
Error: writeimage failed, exiting.
====== Ending image manufacture with error at 20240310-234448
#
 
R

rmsth

New Member
Feb 29, 2024
5
0
1
zcat: write: No space left on device
Click to expand...
Did you set the ramdisk size in the U-Boot?

Code: 
setenv mfg_ramdisk_size 180224
setenv mfg_extra_args ramdisk=262144
 
Last edited:
R

rmsth

New Member
Feb 29, 2024
5
0
1
Hello everyone :)

after successfully converting to MLNX OS, upgrading all the way to 3.6.8012 and connecting CX314A cards on Proxmox a connection could be established with 56GB/s - awesome :-D

What is strange, are the high ping times of ~ 0,5 ms.
Directly connected from card to card they are about 0,06 ms.

Ten times higher via the switch?
Is that normal or is some configuration missing?
 
S

Softlution

New Member
Jul 29, 2022
16
1
3
rmsth said:
Did you set the ramdisk size in the U-Boot?

Code: 
setenv mfg_ramdisk_size 180224
setenv mfg_extra_args ramdisk=262144
Click to expand...
Thanks for your reply. Yes I have done the above. I think the switch got bricked after running /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/image-PPC_M460EX-SX_3.4.0012.img . In the beginning the command didn't go through and I made a path modification to access the image and after accessing the image it bricked the switch
 
R

rmsth

New Member
Feb 29, 2024
5
0
1
Softlution said:
Thanks for your reply. Yes I have done the above. I think the switch got bricked after running /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/image-PPC_M460EX-SX_3.4.0012.img . In the beginning the command didn't go through and I made a path modification to access the image and after accessing the image it bricked the switch
Click to expand...
zcat: write: No space left on device
Click to expand...
Look with du -hs /*, go further down for large files/folders and try to cleanup logs or any other temp files.
 
R

rmsth

New Member
Feb 29, 2024
5
0
1
rmsth said:
Hello everyone :)

after successfully converting to MLNX OS, upgrading all the way to 3.6.8012 and connecting CX314A cards on Proxmox a connection could be established with 56GB/s - awesome :-D

What is strange, are the high ping times of ~ 0,5 ms.
Directly connected from card to card they are about 0,06 ms.

Ten times higher via the switch?
Is that normal or is some configuration missing?
Click to expand...
Optimized Linux and BIOS. Now pings at 0,03 ms over the switch with 48GB/s in iperf :D

The journey with CX3 and now the SX6 has been just a delight.
So happy with these enterprise oldies for pennys :cool:
Wanted to upgrade to 10GBe and now all 40GBe for a fracture of 10GBe Hardware.

Thanks to everyone on building the guides and providing your time and help! :)
 
Last edited:
S

Softlution

New Member
Jul 29, 2022
16
1
3
buck1138 said:
This is for anybody that finds this huge thread and want to skip most of the out of date info and just get a clean updated switch



Hello All, Thought I would post my experience successfully converting my EMC SX6018 using the guide here...


Converting Mellanox EMC switch to SX6012.docx

I originally flashed using the "hacky" guide but this updated guide works much better and gives you a "real" install.

I was able to do the update using a MAC with the built in apache server and tftp server to host the necessary files (It should be equally easy on linux)
www.wirelessphreak.com

Using the Built in TFTP Server on Mac OS

WirelessPhreak.com is my personal blog where I post on topics related to Technology, Security, Travel, and other random stuff I like.
www.wirelessphreak.com www.wirelessphreak.com
You don't need php or anything fancy in apache, just the ability to host files

All the files needed are linked in the above doc.

Once you successfully flash the first image you can just use the web gui to continually update to the next version

I had a few updated versions refuse to start (hung on initializing), when that happened I just booted back to the last version and moved on to the next one and it worked fine. I wonder if I could have jumped directly to the last version but I did all versions except for two.

There is lots of information on this thread about updating fans but here is the best way to control fan speed


Code: 
/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50
/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 2 fan_speed int8 27 set_max uint8 50
/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 3 fan_speed int8 27 set_max uint8 50
/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 4 fan_speed int8 27 set_max uint8 50
/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/PS1/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50
/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/PS2/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50
you can throw it in the /etc/rc.d/rc.local script to start at boot
To do this you need to login to the switch, enable changes, and run # _shell
check where the current mount point is by running
mount
then you can remount the root partition with the following command
mount -o remount,rw /dev/mtdblock7 /

Code: 
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
touch /var/lock/subsys/local

echo "Waiting for clusterd"
COUNT=1
while true
do
    CMD=$(ps aux|grep "/[o]pt/tms/bin/clusterd")
    if [ $? -ne 0 ]; then
        sleep 30
        let COUNT=${COUNT}+1
        if [ $COUNT -gt 10 ]; then
            echo "time out, aborting"
            break
        fi
    else
        sleep 60
        echo "Changing fan speed"
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 2 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 3 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/FAN/FAN" fan_number int8 4 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/PS1/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/PS2/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        break
     fi
done
Click to expand...



HI MATE.
Thanks for sharing.
I tried but I didn't get lucky. Could you please have a look to see if you can find what I am doing wrong? I have an SX6012 and I am not sure if I have PS1 and PS2 fans. I tried removing them but not luck either.


[admin@switch-f34844 ~]# mount
/dev/mtdblock6 on / type jffs2 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/mtdblock8 on /config type jffs2 (rw,noatime)
/dev/mtdblock9 on /var type jffs2 (rw,noatime)
tmpfs on /dev/shm type tmpfs (rw)
tmpfs on /vtmp type tmpfs (rw,size=64M)
[admin@switch-f34844 ~]#
[admin@switch-f34844 ~]# vi /etc/rc.d/rc.local







#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

echo "Waiting for clusterd"
COUNT=1
while true
do
touch /var/lock/subsys/local



echo "Waiting for clusterd"

COUNT=1

while true

do

CMD=$(ps aux|grep "/[o]pt/tms/bin/clusterd")

if [ $? -ne 0 ]; then

sleep 30

let COUNT=${COUNT}+1

if [ $COUNT -gt 10 ]; then

echo "time out, aborting"

break

fi

else

sleep 60

echo "Changing fan speed"

/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_moduu
le string "/FAN/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50

/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_moduu
le string "/FAN/FAN" fan_number int8 2 fan_speed int8 27 set_max uint8 50

/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_moduu
le string "/FAN/FAN" fan_number int8 3 fan_speed int8 27 set_max uint8 50

/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_moduu
le string "/FAN/FAN" fan_number int8 4 fan_speed int8 27 set_max uint8 50

/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_moduu
le string "/PS1/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50

/opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_moduu
le string "/PS2/FAN" fan_number int8 1 fan_speed int8 27 set_max uint8 50

break

fi

done
"/etc/rc.d/rc.local" 64L, 1544C written
[admin@switch-f34844 ~]# chmod +x /etc/rc.d/rc.local
[admin@switch-f34844 ~]# /etc/rc.d/rc.local
Waiting for clusterd
Changing fan speed
Return code: 1
Return message: Action failed
Return code: 1
Return message: Action failed
Return code: 1
Return message: Action failed
Return code: 1
Return message: Action failed
Return code: 1
Return message: Action failed
Return code: 1
Return message: Action failed
[admin@switch-f34844 ~]#
 
S

Softlution

New Member
Jul 29, 2022
16
1
3
metag said:
You are right. Reviewing the log showed that it was just seconds too early. tc overwrote it. Here is the script that finally worked (/etc/rc.d/rc.local, or /mnt/root2/etc/rc.d/rc.local from rootdev mtdblock6
Code: 
#!/bin/sh

touch /var/lock/subsys/local

echo "Waiting for clusterd"
COUNT=1
while true
do
    CMD=$(ps aux|grep "/[o]pt/tms/bin/clusterd")
    if [ $? -ne 0 ]; then
        sleep 30
        let COUNT=${COUNT}+1
        if [ $COUNT -gt 10 ]; then
            echo "time out, aborting"
            break
        fi
    else
        sleep 60
        echo "Changing fan speed"
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN1" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN2" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN3" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN4" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        break
     fi
done
I am waiting for clusterd instead of hwd because that is one of the last daemons launched by pm.
Thanks for your help!
Click to expand...


THANKS FOR sharing.
I have managed to reduce the fan speed but it was still noisy for my home lab. I tried changing fan_speed int8 27 to 20 but the fans starting full speed and stopping every 3 seconds
 
M

Marc_

Always a school day
Jul 22, 2022
67
10
8
Urgh, got most of the way though the conversion but I'm stuck here. I presume it is trying to pull the file from the uniserver but can't access it? Not sure what I did as followed the setup to the letter so far and it's been great. I know it's not the firewall as I just straight turned it off for this:

# /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/image-PPC_M460EX-SX_3.4.0012.img
====== Starting manufacture at 20240320-104429
====== Called as: /sbin/manufacture.sh -a -m ppc -B -u http://192.168.7.77/image-PPC_M460EX-SX_3.4.0012.img

==================================================
Manufacture script starting
==================================================

== Using model: ppc
== Using kernel type: uni
== Using layout: MFL1
== Using partition name-size list:
== Using device list: /dev/mtd
== Using interface list: mgmt0 mgmt1
== Using interface naming: ifindex-sorted
== Smartd disabled
== Cluster enable: no
== Cluster ID: (none)
== Cluster description: (none)
== Cluster interface: (none)
== Cluster master virtual IP address: 0.0.0.0
== Cluster master virtual IP masklen: 0
== Cluster shared secret: (none)
== Cluster expected number of nodes: 0
- Assigning specified interface names in ifindex-sorted order
-- Mapping MAC: B8:CE:F6:68:70:DE from: eth0 to: mgmt0
-- Mapping MAC: B8:CE:F6:68:70:DF from: eth1 to: mgmt1
== Using image from URL: http://192.168.7.77/image-PPC_M460EX-SX_3.4.0012.img

== Calling writeimage to image system
*** Could not retrieve image from http://192.168.7.77/image-PPC_M460EX-SX_3.4.0012.img
Error: writeimage failed, exiting.
====== Ending image manufacture with error at 20240320-104742

EDIT: After some messing around, possible success, it's paused at calling writeimage to image system.

EDIT 2: After an hour and a half I presume teraterm had crashed. Power cycled and it has booted into mellanox os. Logged in, configured using the wizard and it's now been stuck at modules are being configured for 15 mins.

EDIT 3: Fixed that and now have a working switch :)
 
Last edited:
L

larryl79

New Member
Mar 26, 2024
6
0
1
Hi Guys,

I've got a MIS5030 / IS5030 is this switch possible to convert to use ethernet gateway?

Somebody could give me a guide please?!

Thanks
 
You must log in or register to reply here.
Top Bottom