Mellanox Switches - Tips & Tricks
- Thread starter NablaSquaredG
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I haven't used either. I think you can't do anything in Cumulus with out a license. What's so bad with them?
both are just as equally awful.
The last MLNX-OS release to provide support for Switch-IB 2 based platforms is 3.10.4xxx.
The last MLNX-OS release to provide support for Switch-IB based platforms is 3.9.3124.
Is it possible to have access to the installation file X86_64-3.10.4206-installer.bin?
or is there a way to install on ONIE with the .img?
For clarification above. I have sn3700m (hpe version) with ONIE currently installed and need any basic OS to run on this.
Start post updated with table with max firmware versions
On another note:
I've got a couple of 40GBase-LR Lite transceivers (Kaiam XQX2502) that make Mellanox devices (tested SN2700 and ConnectX-5) go crazy (ports blink orange). I have no idea. They work perfectly fine in Brocade devices...
On another note:
I've got a couple of 40GBase-LR Lite transceivers (Kaiam XQX2502) that make Mellanox devices (tested SN2700 and ConnectX-5) go crazy (ports blink orange). I have no idea. They work perfectly fine in Brocade devices...
I think the unmanaged version were missing a management daughterboard that mnakes it impossible
29 watts measured by me for a dual PSU 12-port SX6012 at idle, no transceivers, only mgmt ethernet connected, last firmware, fan speed turned down via rc.local script to around 4500rpm, so just sitting there.
Hi all, I have a pair of SN2010 switches in my lab, they are running Onyx 3.8.2008.
I'd like to use IPv6 MAGP and use BGP in front of a small cluster of k8s nodes running Cillium.
I understand that I need at minimum Onyx 3.9.1000 for using MAGP with IPv6: MAGP
I want to upgrade the switches but they cost me an arm and a leg, so I want to thread carefully
Can anyone confirm whether using the firmware from earlier in this thread will work on these switches ?
I've downloaded the files referenced earlier in this thread: 1.73 GB folder on MEGA
But I see several ways forward. In the zip file, there's a SSDReplacementWithONIENOSInstall.pdf document, it is about installation from USB drive from what I can tell.
But another link in this thread: ESPCommunity is about upgrading from a running system.
Can I just follow this upgrade procedure ? Looking in the downloaded files, when extracted, I see for example:
Upgrade/3.10.4206/image-X86_64-x86_64-x86_64-20230308-191142.tbz
Is that the file I should install by using the:
Will that work ? If so, will any keys or whatnot still be asked ? When I purchased the switches I got them as-is, no further license information, just with what was installed on them.
--
I am contemplating to backup the current ssd so that I may rollback in case things don't work out. I see mention of getting Transcend 452T2 (e.g. TS128GMSA452T2) - I've found this one: https://www.conrad.nl/nl/p/transcen...ata-6-gb-s-retail-ts128gmsa452t2-2302616.html
But this SSD is 128G while currently I have 16G StorFly VSFBM4XC016G-MLX2 installed (see below). Will it not be a problem that the SSD is another size ? Would you suggest to just dd the current disk to an image and then dd the image back onto the new larger SSD ? Or would you advise to put a new clean SSD in the switch and then use the 3.9.3202-installer.bin on usb drive and then upgrade to the 3.10.4206 ?
Thank you so much for the info in this thread, hope you can take some fear away to do the upgrade thanks!
--
Firmware version details:
SSD details:
I'd like to use IPv6 MAGP and use BGP in front of a small cluster of k8s nodes running Cillium.
I understand that I need at minimum Onyx 3.9.1000 for using MAGP with IPv6: MAGP
I want to upgrade the switches but they cost me an arm and a leg, so I want to thread carefully
Can anyone confirm whether using the firmware from earlier in this thread will work on these switches ?
I've downloaded the files referenced earlier in this thread: 1.73 GB folder on MEGA
But I see several ways forward. In the zip file, there's a SSDReplacementWithONIENOSInstall.pdf document, it is about installation from USB drive from what I can tell.
But another link in this thread: ESPCommunity is about upgrading from a running system.
Can I just follow this upgrade procedure ? Looking in the downloaded files, when extracted, I see for example:
Upgrade/3.10.4206/image-X86_64-x86_64-x86_64-20230308-191142.tbz
Is that the file I should install by using the:
Code:
image fetch scp://root:password@server/path-to-image/abovefile.tbz
image install abovefile.tbz
image boot next
configuration write
reload--
I am contemplating to backup the current ssd so that I may rollback in case things don't work out. I see mention of getting Transcend 452T2 (e.g. TS128GMSA452T2) - I've found this one: https://www.conrad.nl/nl/p/transcen...ata-6-gb-s-retail-ts128gmsa452t2-2302616.html
But this SSD is 128G while currently I have 16G StorFly VSFBM4XC016G-MLX2 installed (see below). Will it not be a problem that the SSD is another size ? Would you suggest to just dd the current disk to an image and then dd the image back onto the new larger SSD ? Or would you advise to put a new clean SSD in the switch and then use the 3.9.3202-installer.bin on usb drive and then upgrade to the 3.10.4206 ?
Thank you so much for the info in this thread, hope you can take some fear away to do the upgrade
thanks!--
Firmware version details:
Code:
Product name: Onyx
Product release: 3.8.2008
Build ID: #1-dev
Build date: 2019-11-19 11:15:58
Target arch: x86_64
Target hw: x86_64
Built by: jenkins@27900d252c67
Version summary: X86_64 3.8.2008 2019-11-19 11:15:58 x86_64
Product model: x86onie
BIOS Date: 05/24/2018 12:04:25 Ver: 0ACIF007_02.02.002
Code:
conf t
license install LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
_shell
[admin@switch-080852 ~]# smartctl -a /dev/sda | less
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.15.15-1.el7MELLANOXsmp-x86_64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: StorFly VSFBM4XC016G-MLX2
Serial Number: 41831-0390
Firmware Version: 0202-000
User Capacity: 15,804,137,472 bytes [15.8 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Jan 1 00:18:34 2001 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
...
[admin@switch-080852 ~]# fdisk -l /dev/sda
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
Disk /dev/sda: 15.8 GB, 15804137472 bytes, 30867456 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt
Disk identifier: 80144578-26C9-4862-8A31-7BC967C0E2FA
# Start End Size Type Name
1 2048 6143 2M BIOS boot parti GRUB-BOOT
2 6144 268287 128M unknown ONIE-BOOT
3 524288 786431 128M EFI System primary
4 786432 1048575 128M Microsoft basic primary
5 1048576 1310719 128M Microsoft basic primary
6 1310720 5505023 2G Microsoft basic primary
7 5505024 9699327 2G Microsoft basic primary
8 9699328 10092543 192M Microsoft basic primary
9 10092544 10485759 192M Microsoft basic primary
10 10485760 30867422 9.7G Microsoft basic primaryno.
Rename the .zip from the Mega Folder to .img and you have an original Mellanox Onyx Image. Use this one to upgrade. Do not extract it.
Nope, no licenses needed for Ethernet
just the usual fae cable-stamping-unlock ... stuff
you should replace the SSD anyway, as even with the firmware updates, the StorFly SSDs (or Innodisk) die.
Take out old SSD and perform a clean installation of the 3.9 (or with X86_64-3.10.4206-installer.bin if you can get hold of the file) on the new SSD. You will need a MicroUSB to USB OTG adapter for the USB stick with ONIE and the Mellanox OS installer