Beware of EMC switches sold as Mellanox SX6XXX on eBay

Eong

Member
Dec 24, 2019
35
7
8
So, yes i do understand ... But.
I do understand why clever QDR cables didn't work with FRD10, but i don't understand why stupid cables can work with FRD10.

So basicly the major question - Does anyone have stupid QSFP cable with FDR/56G ?
My DAC cable from mellanox works on 56Gbps under Ethernet mode.
 

crdhn88

New Member
Apr 11, 2020
7
3
3
Is the guide to flash the switch available somewhere? @mpogr was last seen on 18th Jan, and I'm not sure where else to look for it.
 

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
It appears the site is offline now :(

I've mirrored it on IPFS with a couple tweaks to the folder structure & some additions to the instructions file to make it a little more clear; resource ID is QmR8woLoGLPa8kCWDnEeTMNyRK6p6ovY322sGXgd6pWBAe, and you can get to it via the CloudFlare IPFS gateway over here

[EDIT:] OK I gave up on the whole IPFS thing so it's just in an azure storage account with a file browser now

Hopefully that works properly; if you have trouble accessing it let me know and I'll shove it somewhere more 'traditional' :p

Side note: I got my hands on another SX6012 and converted it over just fine! It's a shame about my first one's untimely death, though...
 
Last edited:
  • Like
Reactions: klui

up3up4

Member
Jun 10, 2018
44
10
8
I have a sx6036 mlnx-os corrupted and then follow the conversion guid inject the mlnx-os into partition2. The switch could be temporary run. Is there any way to remanufacture/restore the partition and os to factory original?
 

Rand__

Well-Known Member
Mar 6, 2014
4,551
901
113
There should be factory reset options on cli and gui, you also could try 'upgrading' to the same os level again

Or have you tried that already?
 

up3up4

Member
Jun 10, 2018
44
10
8
I think the sx6036 same with the convered EMC's 6012 roght now, can not use gui "upgrading" anymore. Partition 1 is mini linux, partition 2 is mlnx-os.
 

up3up4

Member
Jun 10, 2018
44
10
8
SX6036 [standalone: master] (config) # image install image-PPC_M460EX-3.6.8012.img

% usage: /sbin/writeimage.sh -m [-u URL] [-f FILE] [-L LAYOUT_TYPE] -d /DEV/N1
[-p PARTNAME -s SIZE] [-O LOCNAME] [-t] [-T LOCNAME]
[-k KERNEL_TYPE] [-w HWNAME] [-I] [-r] [-z] [-y]
[-F LOGFILE] [-v]
usage: /sbin/writeimage.sh -i [-u URL] [-f FILE] [-d /DEV/N1] -l {1,2} [-t]
[-O LOCNAME] [-k KERNEL_TYPE] [-w HWNAME] [-I] [-r]
[-F LOGFILE] [-v]
[-P ID] [-b {1,2}]


Exactly one of '-m' (manufacture) or '-i' (install) must be specified.

Exactly one of '-u' (url) or '-f' (file) must be specified.

-u: specify the URL from which to download the image to install

-f: specify a path in the local filesystem to the image to install
(alternative to -u)

-L LAYOUT_TYPE: STD or other defined layout

-d: device(s) to put image on (i.e. /dev/hda)

-p PARTNAME -s SIZE: (mfg only) partition name and size to use
if desired to be different from ones defined for layout

-O LOCNAME: exclude location (like -BOOT_2, or --all- for all)
-O +LOCNAME: include (normally excluded) location (like +VAR_1)

-B: force install of bootmgr

-l LOCATION_NUMBER: 1 or 2 (install only) specifies which of the two
image locations to which to install

-t: don't use tmpfs for a working area, instead
will attempt to use space in /var

-T LOCNAME: (mfg only) working area is on LOCNAME instead of VAR_1 .
This option is not supported with VPART-based layouts

-k KERNEL_TYPE: uni or smp

-w HWNAME: customer-specific hardware identification string

-I: Ignore any image signature

-r: Require image signature

-F LOGFILE: also log some verbose output to the specified file

-v: verbose mode. Specify multiple times for more output

-P OPER_ID: (install only) track progress using specified progress
operation ID (expected to be already begun)

-b LOCATION_NUMBER: 1 or 2: set the next boot location to the
specified one.

-z : fail imaging if disk is not empty. Currently this fails if
there is a partition table (for msdos and gpt labels), or
for raw (unlabeled) targets, if the first 64k is not all zeros.
Each target to be modified is tested in turn. Only used during
manufacture, not install.

-y : if manufacturing fails, zero out the first 64k of all targets.
In the 64k that follow the first 64k, a failure message will be
written, which may be read by automated manufacturing tools.
Each target to be modified is zeroed in turn. Only used during
manufacture, not install.
 

magictw77

New Member
Jul 27, 2020
1
0
1
I started reading the beginning of this thread and saw that there was an EMC to Mellanox OS conversion guide that needed to be PM'd to get. I am wondering a few things, and sorry if I am sub-threadding a little here, but TLDR:

1. Is the guide still effect and current to the batch of hardware out there being sold; meaning between SwitchX and SwitchX-2 hardwares
2. Can the guide work on any model like the 6036 or the 6018 (looking for internally managed switches, assuming that these are the ones that are affected as the guide updates the management software)
3. If I am wrong on the idea that the management software is the only item that controls the hardware handling of Infiniband and Ethernet sub-functions of the switches; then would it then be applicable to the 6025, 6015 etc...
4. If I decided to use the 6025 with the linux subnet manager able to be installed through some Linux distributions; would those be able to control or offer Ethernet layers while offering Infiniband?

If all the guide still works with the current batch of hardware and is still useful, I would like to put forth that I would require the guide from someone that wants to help me! :)

Both: Sorry in advance and Thank you in advance!
 

Rand__

Well-Known Member
Mar 6, 2014
4,551
901
113
1 - yes
2. It applies to all EMC based 60xx variants with management capabilities (6012/18/36)
3/4. You can run a 60x5 model with a Subnet Managaer if you need an IB switch, you wont have ETH functionality then. I am actually not sure if you can extend a converted switch with a 60x5 for more ETH ports, thats a good question

The guide has been mentioned a few posts up (https://forums.servethehome.com/ind...-as-mellanox-sx6xxx-on-ebay.10786/post-273940 and previous)
 

sparkyblue

New Member
Aug 18, 2019
2
0
1
I'm stuck on step 6.2.B - I am booted into scratch linux (mtdblock6) but I am unsure as to how to adjust admin shell to bin/bash & remove the symlink.

Could someone please describe the lines I would enter to modify that?

Edit: I deleted the symlink /mnt/root2/etc/passwd and created a passwd in /mnt/root2/etc with one line:
admin:x:1000:1000::/:/bin/bash
Can someone verify these steps are correct?
 
Last edited:

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
I'm stuck on step 6.2.B - I am booted into scratch linux (mtdblock6) but I am unsure as to how to adjust admin shell to bin/bash & remove the symlink.

Could someone please describe the lines I would enter to modify that?

Edit: I deleted the symlink /mnt/root2/etc/passwd and created a passwd in /mnt/root2/etc with one line:
admin:x:1000:1000::/:/bin/bash
Can someone verify these steps are correct?
Not quite! Deleting the symlink might not have been the best move, we recommend renaming it so you can move it back later if needed, but it’s not a big deal.

You need to find the file the symlink used to point to - /var/opt/tms/output/passwd, if I recall correctly - and copy it to where the symlink was, then edit the line for the ‘admin’ user so it says /bin/bash instead of /opt/tms/cli.

If you have mtdblock7 mounted to /mnt/root2 and mtdblock9 mounted to /mnt/var, if I’ve remembered the path correctly, this should work:

Bash:
cd /mnt/root2/etc
cp /mnt/var/opt/tms/output/passwd /mnt/root2/etc/passwd
vi /mnt/root2/etc/passwd # edit the file appropriately
Then reboot back into MLNX-OS & when you log in as admin it should drop you to bash.

Also, a useful tip for the recovery OS is to mkdir /bin, /usr/bin, /sbin, and /usr/sbin, then run busybox --install and it’ll dump a bunch of symlinks in for all its various builtins. Makes life a bit easier.
 
Last edited:

sparkyblue

New Member
Aug 18, 2019
2
0
1
Not quite! Deleting the symlink might not have been the best move, we recommend renaming it so you can move it back later if needed, but it’s not a big deal.

You need to find the file the symlink used to point to - /var/opt/tms/output/passwd, if I recall correctly - and copy it to where the symlink was, then edit the line for the ‘admin’ user so it says /bin/bash instead of /opt/tms/cli.

If you have mtdblock7 mounted to /mnt/root2 and mtdblock9 mounted to /mnt/var, if I’ve remembered the path correctly, this should work:

Bash:
cd /mnt/root2/etc
cp /mnt/var/opt/tms/output/passwd /mnt/root2/etc/passwd
vi /mnt/root2/etc/passwd # edit the file appropriately
Then reboot back into MLNX-OS & when you log in as admin it should drop you to bash.

Also, a useful tip for the recovery OS is to mkdir /bin, /usr/bin, /sbin, and /usr/sbin, then run busybox --install and it’ll dump a bunch of symlinks in for all its various builtins. Makes life a bit easier.

Thanks for the instructions for mounting mtdblock9 to fix the passwd file.

I was able to get into bash and run the final few lines:
mst status
flint -i MT_1270110020.bin q
flint --allow_psid_change --override_cache_replacement -d /dev/mst/mt51000_pciconf0 -i /MT_1270110020.bin b

I rebooted and using mtdblock6 reverted the passwd file for admin back to: /opt/bin/tms/cli and was able to login to the switch with admin/admin.

Upon login I see:

MSX6012 login: admin
Password:
Last login: Sat Sep 19 16:00:48 on ttyS0

Mellanox Switch

System is initializing!
This may take a few minutes


Modules are being configured

--

How long does this process usually take? It's been sitting on this for 30-min now with no movement.
 

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
Wait it out. It took a solid 45 minutes on my system, it's something to do with how the partitions are formatted/accessed in recovery vs the normal runtime environment, I think; the first access to any given file takes forever because of some hidden JFFS2 stuff going on in the background

Honestly I'd just leave the shell for 'admin' as /bin/bash (since you can run 'cli' from bash to get into the regular CLI) and go tail -f /var/log/messages to see what it's complaining about
 

francois69380

New Member
Sep 23, 2017
5
0
1
56
Hello,
Thanks again for your very nice work.
I'm trying to upgrade one of my 2 converted MSX6012 to 8012 following guide 1.2.
I have trouble with vpi license not enabled so only ib profile is available, vpi and ethernet do not complete.

The switch was working great with previous kernel install.
Any idea of what I am missing?
 

up3up4

Member
Jun 10, 2018
44
10
8
Hello,
Thanks again for your very nice work.
I'm trying to upgrade one of my 2 converted MSX6012 to 8012 following guide 1.2.
I have trouble with vpi license not enabled so only ib profile is available, vpi and ethernet do not complete.

The switch was working great with previous kernel install.
Any idea of what I am missing?
It seems to be an issue. But you can choose the profile you like when you edit the shell script under etc. Remember you put a number 3 there? Give it a number 4 you will get eth profile.
I will do some further modifications when I have spare time.