Mellanox Switches - Tips & Tricks

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,600
1,056
113
Does anyone have X86_64-3.10.4206-installer.bin and willing to upload?

I need to recover 4 switches and I'd like to avoid the intermediate step via 3.9.3202
 
  • Like
Reactions: Maddox

awedio

Active Member
Feb 24, 2012
777
227
43
Does anyone have X86_64-3.10.4206-installer.bin and willing to upload?

I need to recover 4 switches and I'd like to avoid the intermediate step via 3.9.3202
fyi, sometimes you have to do the intermediate step.
The last switch I had, I tried to go from 3.8.xx to 3.10.xx - no luck!
I've also had a switch where I went straight to 3.10.xx
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,600
1,056
113
fyi, sometimes you have to do the intermediate step.
The last switch I had, I tried to go from 3.8.xx to 3.10.xx - no luck!
I've also had a switch where I went straight to 3.10.xx
That shouldn't apply to clean installs on an empty switch without OS, no?
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,600
1,056
113
Alright, my experiences from flashing 4 SN2700 (Rev B7):

No problems with baudrate. BIOS 0ABZS017_02.02.002 was already installed, was upgraded to 0ABZS017_02.02.004.

The X86_64-3.10.4206-installer.bin didn't work for me at all. I couldn't make it work in any combination.

The ONIE 2020.11-5.3.0005 in legacy boot + X86_64-3.9.3202-installer.bin works pretty well.
Remember to use legacy boot, as UEFI boot will give you a error: disk (hd0,4) not found error for some reason.

PSA: 3.9.3202 seems to be a big buggy, if you don't log in quick enough to run the initial setup it will report that it has lost the connection to the control plane. Also if you plug in the management LAN when you haven't run through initial setup will lockup the switch and requires a reboot...
 
  • Like
Reactions: awedio

up3up4

Member
Jun 10, 2018
88
32
18
Disable csm in bios then boot ONIE 2020.11-5.3.0005 and install X86_64-3.10.4206-installer.bin with new ssd formated in windows the switch is successfully booted with uefi.
 

up3up4

Member
Jun 10, 2018
88
32
18
There is a 32gb innodisk 3ie4 used in windows for new drive speed test. I think onie will format it again while installing the onyx.
 

awedio

Active Member
Feb 24, 2012
777
227
43
That's what I thought. ONIE definitely "re-formats" the disk during the Onyx install.
 

Stephan

Well-Known Member
Apr 21, 2017
1,018
786
113
Germany
Performance question...

On ConnectX3 non-Pro, Linux, ntttcp or nuttcp, 4 streams sender, 4 streams receiver (same box), IP namespace loopback via FDR QSFP+ DAC cable, link speed set to 56 Gbps, I can barely get over 40 Gbps on a 8259CL CPU on an X11SPL-F board. Typical speed is around 39-40 Gbps over the course of a one hour run.

Tried disabling all sleep states of CPU (barely an effect, 1 Gbps or so), I disabled memory init/teardown via boot init_on_alloc=0 init_on_free=0 (a handful of Gbps more), MTU set to 9000 (nothing), various sysctl tweaks (nothing), other tools than ntttcp or nuttcp like iperf/iperf3 way worse (3-10 Gbps less in aggregate), adaptive-rx off rx-usecs 0 tx-frames 64 (a few Gbps more, at best).

What limit exactly am I hitting here?
 

Maddox

New Member
May 28, 2022
13
14
3
Interesting. Seems like this does the trick! ONIE 2020.11-5.3.0005 + X86_64-3.10.4206-installer.bin is now working smoothly.
Glad it works, I also have two switches to prepare, is it possible to have access to the installation file X86_64-3.10.4206-installer.bin?
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,600
1,056
113
Glad it works, I also have two switches to prepare, is it possible to have access to the installation file X86_64-3.10.4206-installer.bin?
I only got it from another member in a PM, so I can't really share it. Maybe they're going to PM you too...
 
  • Like
Reactions: Maddox

linuxsrc

Member
Oct 1, 2018
34
4
8
Brownsburg, IN
I just purchased an MSX1710-BS2F2 and I need to get past version MLNX-OS 3.4.3002 for this switch to enable 56GbE. Could someone share a newer firmware for this?

Thanks
 

linuxsrc

Member
Oct 1, 2018
34
4
8
Brownsburg, IN
I just purchased an MSX1710-BS2F2 and I need to get past version MLNX-OS 3.4.3002 for this switch to enable 56GbE. Could someone share a newer firmware for this?

Thanks
So, I missed the note on not upgrading SX models past 3.6. So I upgraded to 3.9 and things went awry.

I did not equate MSX == SX . As expected the ASIC is no longer available.

The main issue was attempting to find any version that was less than 3.6 and greater than 3.4.12 .

So for now I think this will look good on my server room wall. :)
Good thing I got a decent deal on this, only paid $250.00 for the new wall art.
 
Last edited: