Mellanox Switches - Tips & Tricks
- Thread starter NablaSquaredG
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Just checking-in to this thread and offering a hand. I have almost a decade of experience with mellanox ethernet switches through most generations. I have rebuilt (primary SSD swap) / converted OSes the switches from basic ONIE to ONYX primarily (the the prices is the same for most OSes). I also built a wizard that can create (through simple prompts) a myriad of configs for ONYX specifically.
I have SN2010, SN2100, SN2410, and SN3700c in the field. I do have SX1012s but I don't think about those much since they don't run onie.
Anyway, happy to answer questions. Would love to meet people that might be able to collaborate on some of my own projects (playing with switchdev).
I have SN2010, SN2100, SN2410, and SN3700c in the field. I do have SX1012s but I don't think about those much since they don't run onie.
Anyway, happy to answer questions. Would love to meet people that might be able to collaborate on some of my own projects (playing with switchdev).
I wonder if anyone managed to unlock 200G on SN3700C switches? As those are way easier to get compared to proper 200G SN3700/SN3700M and the only difference in the specs I can see is the CPU in control plane, so probably chip itself is the same across all the product line
I am not sure if the 200G is hardware or software locked. I know on the SN2100B versions (40/56Gbit) we were able to get them to run at 100G with little effort. I can't remember if we popped a cable in and it worked or if we re-flashed. On the other hand, the 56Gbit capability doe require a special QSFP cable and Mellanox cards on the host side that can handle it (plus drivers). I have a couple SN3700Cs out back. If I can procure a 200Gbit cable at some point, I will try it out.
Yeah, 2 out of 3 of my SN2100s are 40G and all of them works with 100G. Not sure if they were reflashed by a seller, or it was unlocked by firmware update.
Need to modify fru info,
Same as usual model, stored on switch board
Another difference is that the number of fans in 200G model is different from that in 100G
Same as usual model, stored on switch board
Another difference is that the number of fans in 200G model is different from that in 100G
Can the fans be set lower for the SN2700? I find that the fans spin down to 5000-6000 rpm but the fans on the power supplies always stay around 10k. There is a certain whine from them. The switch as is running is around 29c so I am guessing if it was running a bit hotter it would not be a problem?
use fae mlxi2c set_fan /PS1/FAN 1 20 Or shell script can reduce fan and ps fan to 20%, if you want below 20% but greater than 4096 or 4500 , than need mod and recompile kernel module ( 20% is hard limit in source code which can mod , 4096 or 4500 is hard limit in firmware which can not mod)
Awesome its the same command as the SX6036 ... will give it a try and see the temps
Thermal Threshold Definitions
docs.nvidia.com
Three thermal threshold definitions are measured by the Spectrum ASICs, and impact the overall switch system operation state as follows:
- Warning – 105°C: On managed systems only: When the ASIC device crosses the 100°C threshold, a Warning Threshold message will be issued by the management software, indicating to system administration that the ASIC has crossed the Warning threshold. Note that this temperature threshold does not require nor lead to any action by hardware (such as switch shutdown).
- Critical – 120°C: When the ASIC device crosses this temperature, the switch firmware will automatically shut down the device.
- Emergency – 130°C: In case the firmware fails to shut down the ASIC device upon crossing its Critical threshold, the device will auto-shutdown upon crossing the Emergency (130°C) threshold.
The last MLNX-OS release to provide support for NVIDIA® SWITCHX® based platforms is 3.6.8012.
The last MLNX-OS release to provide support for Switch-IB based platforms is 3.9.3124.
The last MLNX-OS release to provide support for Switch-IB 2 based platforms is 3.10.4xxx.
The last MLNX-OS release to provide support for Quantum based platforms is 3.12.2xxx.
The last MLNX-OS release to provide support for Switch-IB based platforms is 3.9.3124.
The last MLNX-OS release to provide support for Switch-IB 2 based platforms is 3.10.4xxx.
The last MLNX-OS release to provide support for Quantum based platforms is 3.12.2xxx.
Last edited:
It is in the upgrade folder as its a zip file and you need to rename it as per instructions.
Thats the upgrade file yes, but it's not an installer that can be used with ONIE like the X86_64-3.9.3202-installer.bin can be