EU Mellanox SX6036 36ports QSFP €150

Rand__

Well-Known Member
Mar 6, 2014
4,610
918
113
His conversion replaces files on the switch and is compatible only with a specific version (at least nobody confirmed that others are working)
 

Rand__

Well-Known Member
Mar 6, 2014
4,610
918
113
True, but you're not going to patch that box every other week, and if a newer version is not working you roll back.
You won't be using this for public facing or official company business i assume...

I get the desire but the i don't see the absolute need:)
 

joddlarn

Member
Jul 14, 2018
43
16
8
Sweden
I recieved my unit the other day, works just fine and the updates went through without any problems.

But I was wondering, does anyone know what kind of connector the fans use? The closest I've found is JST-ZH but it isn't quite right.
I'm planning to do some modding to the cooling but don't want to cut the original cables while doing so.
 

Half-Ogre

New Member
Apr 10, 2019
12
3
3
Consider changing thermal interface at main cpu (cant miss it, its largest radiator on board).Manufacturing date is 2013 and it just dried out.
My whole lab is waiting till i fix ethernet mode on those, contacted few members few days ago, but no luck yet,PM me if someone know how to achieve that.Otherwise very nice devices and "great deal" for sure.
 

JTF195

New Member
Nov 15, 2017
9
6
3
30
Due to the legal gray areas involved, it's not really safe to hand out the keys to the castle, but here are a few tips:
1. Locate, download, and unpack a MLNX_OS image (Preferably the x86 variant)
2. Locate the genlicense and dumplicense executables in the unpacked image.
3. Play around with them (try using ltrace :))

The past few pages of the thread have more detailed explanations
 

MSX-fan

New Member
May 2, 2019
7
0
1
That is what I did on my first switch (just upload the latest version via web GUI) and it was much, much faster :)
It's just that it seems to be running under high CPU load (about 2.5 all the time) and I can't find the reason, so I decided to follow the upgrade procedure for the second one and see if it makes any difference (still 2 more versions to go before finding out if it did help, I will share the results :) ).
Could this reported High CPU load (because of problem in the way some of the managed switches do make logging), be something to explain the CPU load??
HPE Support document - HPE Support Center
 

iJKCode

New Member
Apr 24, 2019
5
2
3
25
2. Locate the genlicense and dumplicense executables in the unpacked image.
3. Play around with them (try using ltrace :))
Thanks a lot for that info, dumplicense really is the key to everything :).

Could this reported High CPU load (because of problem in the way some of the managed switches do make logging), be something to explain the CPU load??
HPE Support document - HPE Support Center
I am using the switch in ethernet only mode and also see an average cpu usage of 50% with peaks up to 90%.
I tried to set the log level to warning instead of notice but did not see a decrease in cpu usage.

Logging in to the switch using sftp actually allows you to browse the filesystem, but many directories are not accessible.
The /var/opt/tms and /var/log directories are accessible though and contains some interesting stuff.
I noticed that the /var/log/messages file logs the top command output when it detects increased cpu usage:

Code:
[health.WARNING]: CPU Utilization raising detected top output flowing:
[health.NOTICE]: top: top - 19:42:21 up  5:37,  2 users,  load average: 5.39, 3.83, 3.29
[health.NOTICE]: top: Tasks: 103 total,   3 running, 100 sleeping,   0 stopped,   0 zombie
[health.NOTICE]: top: Cpu(s): 36.2%us, 10.3%sy,  0.0%ni, 52.4%id,  0.0%wa,  0.0%hi,  1.1%si,  0.0%st
[health.NOTICE]: Mem:   2076104k total,  1151420k used,   924684k free,        0k buffers
[health.NOTICE]: Swap:        0k total,        0k used,        0k free,   376388k cached
[health.NOTICE]:   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
[health.NOTICE]:  4372 admin     20   0  261m  83m  20m R 45.9  4.1  17:37.99 mgmtd             
[health.NOTICE]:  5158 admin     20   0  290m 6728 4480 S 11.9  0.3   6:13.27 metad             
[health.NOTICE]:  5157 admin     20   0  269m  29m 4800 S  6.8  1.5   0:03.69 mlag               
[health.NOTICE]:  5223 admin     20   0  774m 364m  16m S  6.8 18.0  14:18.25 iss               
[health.NOTICE]:  1775 admin     30  10     0    0    0 R  5.1  0.0   2:16.40 jffs2_gcd_mtd9     
[health.NOTICE]:  4944 admin     20   0  155m 4896 3572 S  3.4  0.2   0:03.63 chad               
[health.NOTICE]:  5207 admin     20   0  396m 100m  10m S  3.4  4.9  24:25.16 sx_sdk             
[health.NOTICE]: 22426 admin     20   0  3056 1296 1136 S  3.4  0.1   0:00.02 logrotate_wrapp   
[health.NOTICE]:  5201 admin     20   0  170m 4372 3036 S  1.7  0.2   3:08.68 acld               
[health.NOTICE]: 18078 admin     20   0  183m  64m  25m S  1.7  3.2   0:16.69 cli               
[health.NOTICE]: 22423 admin     20   0  2640 1004  792 R  1.7  0.0   0:00.03 top
So the switch management daemon is most likely the culprit.

This is all I found for now.
Kr,
Jens

PS: I'm new here so please excuse me if i'm doing anything against the rules.
 
  • Like
Reactions: int0x2e

iJKCode

New Member
Apr 24, 2019
5
2
3
25
Also FYI using these commands i was able to remove the uboot password (mlnx-os 3.6.8010):

For recent MLNX-OS (3.6.6003 for example, 3.6.4006 will not work) it's even easier:

Code:
enable
configure terminal
boot bootmgr password 7 ""
write memory
show bootvar
There should be no passwords in boot manager, IMO.
Even Huawei allows bootmgr access:
[All About Switches] 07 User Login Password-Switch-Huawei Enterprise Support Community

Code:
/opt/tms/bin/mddbreq /config/db/initial set modify - /system/bootmgr/password string ''
eetool -a bf -s UBPASSWD=""
That makes settings persistent and on the next boot the password is gone.


If that helped, please endorse: https://goo.gl/RfjbnG
Thanks @necr
 
  • Like
Reactions: joddlarn

hlhjedsfg

Member
Feb 2, 2018
37
7
8
30
Thank's all for the deal and the shared knowledge. Mine is now running with ethernet, thank's to @JTF195 for the hint !

Thank's also to @necr to bypass the u-boot password (initialy try to set the shared secret with mddbreq after removing the u-boot password and booting in single user) this can be usefull for other purpose.

Does the 40Gb/s DAC from FS.com permit 56Gbe ?
 

Half-Ogre

New Member
Apr 10, 2019
12
3
3
Got 56gbe ethernet mode with original mellanox cable and connectX3 pro cards pn: MC220731V-003 3m.Also got dell breakout cable from ebay JR54Y Dell QSFP+ to 4x SFP+ Breakout Cable 7 Meters DAC-QSFP-4SFP-10G-7M and it works fine with 10gbe connectX2 card and qlogic dualport adapters.Split port needs to be set from command line, after it shows fine in port web gui section.Have not tested yet actual speed rates, just negotation rates shows 10gbe. Original cables are AOC not copper ones, its important latency wise.
 

iJKCode

New Member
Apr 24, 2019
5
2
3
25
I use FS DACs in combination with connectx3 fdr nics and they only go up to 40G ethernet.
Though i think that that is a limitation of the nic.
I can check the advertised link modes from the switch when i get home later using ethtool if you want.
 

hlhjedsfg

Member
Feb 2, 2018
37
7
8
30
Got 56gbe ethernet mode with original mellanox cable and connectX3 pro cards pn: MC220731V-003 3m.Also got dell breakout cable from ebay JR54Y Dell QSFP+ to 4x SFP+ Breakout Cable 7 Meters DAC-QSFP-4SFP-10G-7M and it works fine with 10gbe connectX2 card and qlogic dualport adapters.Split port needs to be set from command line, after it shows fine in port web gui section.Have not tested yet actual speed rates, just negotation rates shows 10gbe. Original cables are AOC not copper ones, its important latency wise.
Thank's for the info. The MC220731V-003 is listed at 130€ on FS.COM, but i found a DAC advertised at 56Gb (miss it before, must search 56G, not 56GB or 56GBE) :
Câble QSFP+ 56G DAC Mellanox MC2207130-001 Compatible
36€ for 1M seems good.

I'll give a try and get back if it works !