Beware of EMC switches sold as Mellanox SX6XXX on eBay

metag

Member
Apr 26, 2016
63
12
8
You could try booting into mtd6 and enter the commands in /mnt/root2/etc/initi.d/rcS and placing commands there. You might need to wait until the system stabilizes so some of the daemons are running.
Thanks for the hint! Will try it and report back.
Off chance did you install 8012? Could you check if VLAN commands work per my post at https://forums.servethehome.com/ind...-as-mellanox-sx6xxx-on-ebay.10786/post-279407 ?
Just tried. I have the same result as you. "show vlan" and "vlan ###" are not recognized. I got curious and tried to create VLAN on webgui. It worked there. I was able to create VLAN and assign ports to it. But cli still says no recognized.
 
Last edited:
  • Like
Reactions: klui

klui

Active Member
Feb 3, 2019
194
82
28
Thanks for confirming and posting your progress here. I guess that's one of the "issues" @Eong was referred to. I've been wanting to try setting the switch to eth mode only on another install but can't get off my butt to start. It takes around 2.5-3 hours and I just don't have the time recently.
 

metag

Member
Apr 26, 2016
63
12
8
Another issue I noticed but saw no one mentioning: only one PSU is listed either from cli or webgui. I guess this warning in the log has something to do with it:

Oct 15 16:03:52 MSX6012S1 hwd[5090]: TID 1208143072: [hwd.WARNING]: adjust_pses_number_according_to_machine: P/N:[(null)] for Dingo is unknown. Treating it as 1PSU Dingo

Clarify context: this is on 3.6.8012. Two PSUs were listed on 3.6.1002.
 

klui

Active Member
Feb 3, 2019
194
82
28
I don't have messages like that.

Are they both plugged in? What about the LEDs? The LEDs and alerts behave better under 8012. Under 1002, there is an alert and its LED is red if only one PSU is connected to power.

Under 8012, there is no alert in that situation. Only the PSU LED is off and alert is not red.
 

metag

Member
Apr 26, 2016
63
12
8
I don't have messages like that.

Are they both plugged in? What about the LEDs? The LEDs and alerts behave better under 8012. Under 1002, there is an alert and its LED is red if only one PSU is connected to power.

Under 8012, there is no alert in that situation. Only the PSU LED is off and alert is not red.
Hmm...that is strange. I swear both PSUs showed up in 1002. So I assume it was a side effect of 8012. Yes, both of them are plugged in. And yes, all LEDs are green. And I have two 6012s. They both are on 8012 and they both behave like that...
 

klui

Active Member
Feb 3, 2019
194
82
28
I take that back. I just powered mine up and I see the same thing with both PSUs powered up.
 

metag

Member
Apr 26, 2016
63
12
8
You could try booting into mtd6 and enter the commands in /mnt/root2/etc/initi.d/rcS and placing commands there. You might need to wait until the system stabilizes so some of the daemons are running.
I tried this and it had no effect. I then tried rc.local and it shows something like hwd is not running. How do I find out when hwd finishes loading?
 

klui

Active Member
Feb 3, 2019
194
82
28
You can try something Q&D like this
Code:
COUNT=1
while [ $COUNT -le 3 ]
do
    CMD=$(ps aux|grep '/[o]pt/tms/bin/hwd')
    if [ $? -ne 0 ]; then
        sleep 30
        let COUNT=COUNT+1
    else
        echo 'ok'
        #  Run the commands here after hwd is running
     
     
        break
    fi
done
Play around with the sleep and the -le 3 commands.

Oh, one more thing. I'm now noticing these in /var/log/debug. Ports work though, but I've only use ethernet.

rh[5527]: [web.WARNING]: util_ui_api_is_ib_eth_vpi_enabled: HW supports: IB ETH VPI profiles, but they are not enabled by HW VPD or license
 
Last edited:

metag

Member
Apr 26, 2016
63
12
8
You can try something Q&D like this
Code:
COUNT=1
while [ $COUNT -le 3 ]
do
    CMD=$(ps aux|grep '/[o]pt/tms/bin/hwd')
    if [ $? -ne 0 ]; then
        sleep 30
        let COUNT=COUNT+1
    else
        echo 'ok'
        #  Run the commands here after hwd is running
   
   
        break
    fi
done
Play around with the sleep and the -le 3 commands.

Oh, one more thing. I'm now noticing these in /var/log/debug. Ports work though, but I've only use ethernet.

rh[5527]: [web.WARNING]: util_ui_api_is_ib_eth_vpi_enabled: HW supports: IB ETH VPI profiles, but they are not enabled by HW VPD or license
Thanks, the script worked with minor changes. But it had no effect on fan speeds. I even tried to check for the cablemond daemon instead of hwd because cablemond is launched by pm after hwd according to the log. I am giving up now and use the tc hack if I want a permanent solution.

Yes, I saw those warnings in my logs too, a lot actually.
 

klui

Active Member
Feb 3, 2019
194
82
28
There's probably an interplay between different daemons. The affect of hard coded tc values don't take effect until maybe several minutes after the kernel is booted. If the commands work after the switch idles that means you just need to wait longer before running the commands. Like wait 60 seconds after the process shows up.
 

metag

Member
Apr 26, 2016
63
12
8
There's probably an interplay between different daemons. The affect of hard coded tc values don't take effect until maybe several minutes after the kernel is booted. If the commands work after the switch idles that means you just need to wait longer before running the commands. Like wait 60 seconds after the process shows up.
You are right. Reviewing the log showed that it was just seconds too early. tc overwrote it. Here is the script that finally worked (/etc/rc.d/rc.local, or /mnt/root2/etc/rc.d/rc.local from rootdev mtdblock6
Code:
#!/bin/sh

touch /var/lock/subsys/local

echo "Waiting for clusterd"
COUNT=1
while true
do
    CMD=$(ps aux|grep "/[o]pt/tms/bin/clusterd")
    if [ $? -ne 0 ]; then
        sleep 30
        let COUNT=${COUNT}+1
        if [ $COUNT -gt 10 ]; then
            echo "time out, aborting"
            break
        fi
    else
        sleep 60
        echo "Changing fan speed"
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN1" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN2" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN3" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        /opt/tms/bin/mdreq action /system/chassis/actions/set-fan-speed fan_module string "/MGMT/FAN4" fan_number int8 1 fan_speed int8 27 set_max uint8 50
        break
     fi
done
I am waiting for clusterd instead of hwd because that is one of the last daemons launched by pm.
Thanks for your help!
 
  • Like
Reactions: klui

Sjhwilkes

New Member
Oct 17, 2020
3
0
1
So before I pull the trigger on a pair of these for my lab - can someone config the SX1012 and SX6012 are the same thing right? Just Ethernet v Infiniband out of the box, and switchable anyhow.
 

sh1

New Member
Sep 20, 2020
2
1
3
FYI: Using initramfs minimal root image, SX6012 can be booted off tftp. No need to flash a minimal linux image onto the device => less likely to brick the device. In fact, initramfs can be used to boot off a usb drive. Currently using SSD with a usb SATA adapter to boot a SX6012 for testing purpose.