Mellanox Switches - Tips & Tricks

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Rand__

Well-Known Member
Mar 6, 2014
6,643
1,778
113
enable breakout mode if you want to use more than the first sfp+ head?
(or sfp28, depending on whether we talk about a 100G or 40/56G switch)
(and depending on breakout cable used, dual/quad, generation)
 

up3up4

Member
Jun 10, 2018
88
32
18
After the SN2700 DC power supply is replaced with a Liteon AC power supply, Power Supply 1/2 is unresponsive. How can I read/modify the i2c/pmbus information of the power supply?
I think the original psu is also made by Delta. I inserted two Delta DPS-750XB-A in the switche and finally those power supplys can be recognized.
 

bitbckt

will google compiler errors for scotch
Feb 22, 2022
216
136
43
After the SN2700 DC power supply is replaced with a Liteon AC power supply, Power Supply 1/2 is unresponsive. How can I read/modify the i2c/pmbus information of the power supply?
I've been wondering about this too, so I went poking... this looks interesting, but requires xxd:

Code:
/bin/hw-management-read-ps-eeprom.sh
Based on the usage hint at the bottom of that file, it looks like we can read EEPROM data from the PSUs here:

Code:
# ls /var/run/hw-management/eeprom/       
cpu_info   fan2_info  fan4_info  psu1_vpd   psu2_vpd
fan1_info  fan3_info  psu1_info  psu2_info  vpd_info
And update the firmware using the vendor-specific scripts:

Code:
hw_management_psu_fw_update_common.py  hw_management_psu_fw_update_murata.py
hw_management_psu_fw_update_delta.py
I'm looking into where the firmware for the PSUs might live. It would be nice to pull it off of a legitimate Mellanox PSU and try flashing it on to a replacement OEM Delta PSU. That might address the fan speed issue referenced elsewhere in this thread.
 
  • Like
Reactions: up3up4

theDXT

New Member
Apr 19, 2023
5
2
3
Calgary, AB, Canada
thedxt.ca
After you have the above files, you will be able to replace the SSD on the device at will, but the non-original SSD will not make the SSD health report take effect.

Please also be careful not to try to use UEFI boot to install these on the SN2××× device, the system will think that the DMI information is incorrect, and then try to update the BIOS to restart continuously, although ONIE and the system itself support UEFI.
UEFI booting may be available on newer devices, but I haven't tried it.

The way to enter the BIOS settings is to press Ctrl+B during booting. If you are prompted to ask for a password, it may be admin.
If the password admin doesn't work you can reset the BIOS back to default without knowing the password by doing the following
  1. Code:
    license install LK2-RESTRICTED_CMDS_GEN2-88A1-NEWD-BPNB-1
  2. Code:
    _shell
  3. Code:
    opt/tms/bin/bios_update.sh default
  4. let the switch reboot twice then the password should be admin
This worked for me on the SN2410 which I didn't have the BIOS password for.
 

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,600
1,056
113
anyone has 3.10.4206? installer or update image?
according release notes, this version fix boot i2c init and ntp issue, important for everyone
Internal Ref.CategoryDescriptionFixed in Version
3167994MAGP
In the event of a network loop, traffic with
MAGP source vMAC may be forwarded back
to the switch port.
3.10.42xx
3315167NTP, LoggingNTPD prints the following log every minute:
ntpd5510: bind(23) AF_INET6 ::1#123 flags
0×5 failed: Cannot assign requested
address
3.10.42xx
3338504NTP, VRFWhen enabling VRF management on NTP,
the NTP servers are not displayed.
3.10.42xx
3362827OpenFlowWhen OpenFlow flow is modified with a
group that contains more than 50 actions,
the switch crashes.
3.10.42xx
3284730PTP, Sysdump,
Output
Output of show PTP commands was
missing in sysdump.
3.10.42xx
 
  • Like
Reactions: dbTH

dbTH

Member
Apr 9, 2017
152
60
28
Internal Ref.CategoryDescriptionFixed in Version
3167994MAGP
In the event of a network loop, traffic with
MAGP source vMAC may be forwarded back
to the switch port.
3.10.42xx
3315167NTP, LoggingNTPD prints the following log every minute:
ntpd5510: bind(23) AF_INET6 ::1#123 flags
0×5 failed: Cannot assign requested
address
3.10.42xx
3338504NTP, VRFWhen enabling VRF management on NTP,
the NTP servers are not displayed.
3.10.42xx
3362827OpenFlowWhen OpenFlow flow is modified with a
group that contains more than 50 actions,
the switch crashes.
3.10.42xx
3284730PTP, Sysdump,
Output
Output of show PTP commands was
missing in sysdump.
3.10.42xx
3.10.4206 helped in fixing the NTP issue on mangement VRF in my case. Highly recommended to update to this release whenever you can
 

bitbckt

will google compiler errors for scotch
Feb 22, 2022
216
136
43
I'm looking into where the firmware for the PSUs might live. It would be nice to pull it off of a legitimate Mellanox PSU and try flashing it on to a replacement OEM Delta PSU. That might address the fan speed issue referenced elsewhere in this thread.
Looks like PSU fan speed control is available via PMBus: OpenNetworkLinux/driver-hwmon-pmbus-dni_dps460.patch at master · opencomputeproject/OpenNetworkLinux

Quoting the patch for the record:

Code:
+/* Data provided by DELL Inc */
+#define FAN_RPM_MIN 7200
+#define FAN_RPM_MAX 18000
 

Serven

New Member
May 2, 2023
2
0
1
I have an SB7700, I believe I need a license to get 100gb EDR instead of the 56gb FDR correct? Where can I buy something like this? Will Nvidia still sell it?
 

Serven

New Member
May 2, 2023
2
0
1
Ah you know I am using Mikrotik cables while I wait for the official Mellanox ones to arrive. That could be maybe what's causing it to limit down to 56. I will test with the Mellanox ones when they arrive and report back. Thank you didn't think of that.

Don't have access to the unit right now but it is running mlnx-os 3.8 I believe. I want to update to 3.9 but first waiting for a storage drive too because I am worried I break things.
 

linuxsrc

Member
Oct 1, 2018
34
4
8
Brownsburg, IN
I am looking at a couple of different SX1710 switches on Ebay. Is there only one version of this switch? I see at least 3 different names for this product listing

SX1710
SX1710 IB SwitchX-2
SX1710 MSX1710-BS2F2
SX1710 MSX1700-BS1F2

Details on the differences are hard to come by, if someone could direct me to some info on these, I would appreciate it.

Also, what are the differences between the SX1700 and the SX60306? Trying to determine if the cost differences are worth it.

Thanks