Mellanox Switches - Tips & Tricks

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

dbTH

Member
Apr 9, 2017
153
60
28
take out the SSD and use dd on Linux
Thanks for the reply.

Yeah, that's the last resort that I would use since that would require the SSD physically taken out of switch. I was wondering if there's a live cloning tool available ( similar to few avilable for Linux OS image cloning) or if someone have written a tool that can just do "image create" ?
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,617
1,071
113
You may be able to create a full SSD image, but definitely not something you can use with the image commands (like OS updates)
 

SadoKitten

Active Member
Apr 26, 2018
102
43
43
iowa
ezsolutions.us
I got onyx loaded onto my s2700, but it appears the web interface is disabled. I looked through the mellanox documentation but it shows as enabled.so i must be missing a step.
 

klui

༺༻
Feb 3, 2019
924
530
93
If you have a PXE server you should be able to PXE boot into Clonezilla.

I've successfully cloned entire disks from open enterprise switches using Clonezilla. Use an appropriate adapter as I don't have any Mellanox SNnnnn switch. They may have switched to M.2 form factor. The switches I have use mSATA so I use this variety https://www.ebay.com/itm/151466719602--no screws to mess with.
 

SadoKitten

Active Member
Apr 26, 2018
102
43
43
iowa
ezsolutions.us
Ok so i have the s2700 up and running with the onie and X86_64 3.9.3202 . I have the update of X86_64-3.10.4206-installer.bin but it will not install it. It complains that the bios is out of date. Does anyone have the latest bios for it?
 

nasbdh9

Active Member
Aug 4, 2019
180
114
43
Ok so i have the s2700 up and running with the onie and X86_64 3.9.3202 . I have the update of X86_64-3.10.4206-installer.bin but it will not install it. It complains that the bios is out of date. Does anyone have the latest bios for it?
currently no updated BIOS for the SN2xxx series

X86_64-3.10.4206-installer.bin is used for the new installation system in onie

If you want to update the system, the required file is onyx-X86_64-3.10.4206.zip
This link explains how to do it: ESPCommunity (replace image fetch scp in the text with http, because scp is too slow)
 

dbTH

Member
Apr 9, 2017
153
60
28
If you have a PXE server you should be able to PXE boot into Clonezilla.

I've successfully cloned entire disks from open enterprise switches using Clonezilla. Use an appropriate adapter as I don't have any Mellanox SNnnnn switch. They may have switched to M.2 form factor. The switches I have use mSATA so I use this variety https://www.ebay.com/itm/151466719602--no screws to mess with.
Clonezilla via PXE boot is a good option if your switch can be configured to PXE boot. But I don't know if Mellanox SNxxxx switch bios can be configured to PXE boot. Does anyone know if it can?
 
Last edited:

klui

༺༻
Feb 3, 2019
924
530
93
That's a great question. It took a little bit of web searching but while I was not able to find how to PXE boot Mellanox switches I thought about ONIE being the launch pad and found


where it describes recompiling converting the Clonezilla ISO into an ONIE installer "NOS".

Open switches that have a traditional BIOS from, say AMI, will do PXE no problem but it's an avenue I can't fully recommend because I have bricked some switches during the process. One temporarily and another kinda "permanently"--I think it's because I was messing with a BIOS setting on an unrelated experiment but thought I'd fully disclose that. Since I have multiples of that same SKU I'm thinking of using a cheap Ch341a mini-programmer to attempt to compare potential corrupted SPI chips.
 
Last edited:

dbTH

Member
Apr 9, 2017
153
60
28
That's a great question. It took a little bit of web searching but while I was not able to find how to PXE boot Mellanox switches I thought about ONIE being the launch pad and found


where it describes recompiling converting the Clonezilla ISO into an ONIE installer "NOS".

Open switches that have a traditional BIOS from, say AMI, will do PXE no problem but it's an avenue I can't fully recommend because I have bricked some switches during the process. One temporarily and another kinda "permanently"--I think it's because I was messing with a BIOS setting on an unrelated experiment but thought I'd fully disclose that. Since I have multiples of that same SKU I'm thinking of using a cheap Ch341a mini-programmer to attempt to compare potential corrupted SPI chips.
That document is a good starting point. Need a cheaper switch with ONIE to experiment that. Won't be able do that with Mellanox SN2xxx at this point as such switch is at premium.
 

Freebsd1976

Active Member
Feb 23, 2018
404
73
28
dd over ssh is another way, and it is more simple then clonezilla
Code:
 dd if=/dev/sda | gzip -1 - | ssh root@192.168.1.73 dd of=sn2100.gz
 

nasbdh9

Active Member
Aug 4, 2019
180
114
43
Anyone know if these can be converted to ethernet? New Mellanox 36-port MSX6790 SX6790 QSFP QDR InfiniBand Switch EMC OEM 2 x PSU | eBay

Edit: No they are unmanaged EMC switches
MSX6790 is an SX2 switch using the x86 control plane, although the current x86 system image still retains the SX2 firmware.
But I don't know if MSX6790 uses onie or something else to boot the system, or he really only supports up to 3.6.8012 like a device using PPC control plane


Ignoring the above, for these EMC products they do not contain any control plane, that is, they do not have any operating system
 
Last edited:

awedio

Active Member
Feb 24, 2012
779
228
43
What USB to Serial adapter do you guys use to connect your laptop to the Mellanox switch serial console (especially the SN2xxx series)? There's an article that talked about the adapter selection (5 Steps for Selecting the Right USB to Serial adapter). But not sure if it is really worth to buy those recommended. I have an older adapter (pre-2012) with older Prolific PL2303HXA chip lying around , however, the adapter driver is no more compatible with the newer Windows OS. Looking to get an adapter with newer chip, and would like to hear your suggestions.
Gearmo USB to Serial RS-232 Adapter with LED Indicators

Also had good luck with the GUC232A from IOGear
 
Last edited:
  • Like
Reactions: dbTH

awedio

Active Member
Feb 24, 2012
779
228
43
Curious question...anyone have any luck with 115200 baud rate?
9600 is all I can get to work consistently.
 

dbTH

Member
Apr 9, 2017
153
60
28
Curious question...anyone have any luck with 115200 baud rate?
9600 is all I can get to work consistently.
Mine works at 115200 baud rate even with the older Prolific PL2303HXA chip serial adapter. But it also depends on your switch hardware version I guess.
 

awedio

Active Member
Feb 24, 2012
779
228
43
Mine works at 115200 baud rate even with the older Prolific PL2303HXA chip serial adapter. But it also depends on your switch hardware version I guess.
That's an interesting angle I never even considered.
I'll have to try with my Gearmo.
The IOGear is what I'm currently using.
 

awedio

Active Member
Feb 24, 2012
779
228
43
Messing around with a SN2700 o/s re-install.
I keep getting this error.
Any ideas?

Code:
Error: Partition doesn’t exist.

== Creating ext3 filesystem on /dev/sda10 for VAR

mke2fs: No such device or address while trying to determine filesystem size

*** Could not make filesystem on /dev/sda10

== Cleanup
 

ano

Well-Known Member
Nov 7, 2022
702
301
63
sn2700s are funny, they want 9600, but update them, and new onyx wants 115200... but get into old bios.. yeah 9600
 
  • Like
Reactions: awedio