Beware of EMC switches sold as Mellanox SX6XXX on eBay

klui

Active Member
Feb 3, 2019
160
67
28
Your logs don't show any critical errors. Did I miss something?

If you've followed the guide and your loglevel is set to 2 you won't see ECC errors during boot. The odd thing for me is the recovery partition won't show any kernel messages with loglevel set to 2.

Maybe for you a reinstall is in order.
 

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
My bad, I forgot to even mention the actual issue ‍*facepalm* it refuses to identify any optics/DACs; they all show as "unsupported module" or "invalid/missing EEPROM data in module".

I did once manage to get it to identify a genuine Mellanox 40G IB DAC, but it won't even identify that anymore, or any SFP+ modules installed in a QSA. It also takes it a solid two or three minutes to load the sx6012 [standalone: master] > prompt after logging in.

The boot log doesn't look any different with loglevel=4 either, guess I should try five or six, or just accept that the blown fan chip's messed up some other stuff too.

[Edit:] I would really, really appreciate a known-good switch's printenv output, though. If anyone feels like passing theirs on, feel free to remove any values that identify the specific switch if that concerns you - placeholders like <serial> or <mac> are totally fine.
 
Last edited:

klui

Active Member
Feb 3, 2019
160
67
28
I would really, really appreciate a known-good switch's printenv output, though. Feel free to remove any values that identify the specific switch, if you're concerned - a placeholder like <serial> or <mac> is totally fine.
Here's mine. I just x'ed out ethaddr and eth1addr MACs. Not sure how to turn off emojis. addip is "s e t e n v . . . { n e t d e v } : o f f p a n i c = 1"

=> printenv
bootcmd=run flash_jffs2
baudrate=9600
loads_echo=
hostname=mlnx460ex
netdev=eth0
nfsargs=setenv bootargs root=/dev/nfs rw nfsroot=${serverip}:${rootpath}
ramargs=setenv bootargs root=/dev/ram rw
addip=setenv bootargs ${bootargs} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}:${netdev}:eek:ff panic=1
addtty=setenv bootargs ${bootargs} console=ttyS0,${baudrate}
addmisc=setenv bootargs ${bootargs}
initrd_high=30000000
kernel_addr_r=400000
fdt_addr_r=800000
ramdisk_addr_r=C00000
hostname=mlnx460ex
ramdisk_file=mlnx460ex/uRamdisk
rootpath=/opt/eldk/ppc_4xxFP
flash_self=run ramargs addip addtty addmisc;bootm ${kernel_addr} ${ramdisk_addr} ${fdt_addr}
flash_nfs=run nfsargs addip addtty addmisc;bootm ${kernel_addr} - ${fdt_addr}
net_nfs=tftp ${kernel_addr_r} ${bootfile}; tftp ${fdt_addr_r} ${fdt_file}; run nfsargs addip addtty addmisc;bootm ${kernel_addr_r} - ${fdt_addr_r}
net_self_load=tftp ${kernel_addr_r} ${bootfile};tftp ${fdt_addr_r} ${fdt_file};tftp ${ramdisk_addr_r} ${ramdisk_file};
net_self=run net_self_load;run ramargs addip addtty addmisc;bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r}
fdt_file=mlnx460ex/mlnx460ex.dtb
flash_self_old=run ramargs addip addtty addmisc;bootm ${kernel_addr} ${ramdisk_addr}
flash_nfs_old=run nfsargs addip addtty addmisc;bootm ${kernel_addr}
net_nfs_old=tftp ${kernel_addr_r} ${bootfile};run nfsargs addip addtty addmisc;bootm ${kernel_addr_r}
load=tftp 200000 mlnx460ex/u-boot.bin
update=protect off 0xFFFA0000 FFFFFFFF;era 0xFFFA0000 FFFFFFFF;cp.b ${fileaddr} 0xFFFA0000 ${filesize};setenv filesize;saveenv
upd=run load update
dhcp_vendor-class-identifier=bootmfg:hwname:mlnx460ex:
clear_filesize=setenv filesize
mfg_dir=mlnx460ex
mfg_args=setenv bootargs root=/dev/ram rw ramdisk_size=${mfg_ramdisk_size} ${mfg_extra_args}
mfg_common_args=run addtty addmisc
mfg_load=tftp ${kernel_addr_r} ${mfg_root}${mfg_dir}/${mfg_kernel_file};tftp ${fdt_addr_r} ${mfg_root}${mfg_dir}/${mfg_fdt_file};tftp ${ramdisk_addr_r} ${mfg_root}${mfg_dir}/${mfg_ramdisk_file}
mfg_nodhcp=echo "Manufacture will TFTP from directory ${mfg_root}${mfg_dir}, and boot";echo; run clear_filesize ; run mfg_load;if test 0${filesize} -gt 0; then echo Booting mfg ; run mfg_args mfg_common_args;bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ; else ; echo Failed mfg load ; fi
mfg=echo "Manufacture will DHCP, TFTP from directory ${mfg_root}${mfg_dir}, and boot";echo; dhcp; run clear_filesize ; run mfg_load;if test 0${filesize} -gt 0; then echo Booting mfg ; run mfg_args mfg_common_args;bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ; else ; echo Failed mfg load ; fi
menu_file=menu.img
menu_load=tftp ${menu_addr_r} ${mfg_root}${mfg_dir}/${menu_file}; if test $? -ne 0; then run clear_filesize ; echo Download failed ;fi
menu_usb_load_ext2=usb start; ext2load usb ${mfg_usb_dev}:${mfg_usb_part} ${menu_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${menu_file};
menu_usb_load_fat=usb start; fatload usb ${mfg_usb_dev}:${mfg_usb_part} ${menu_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${menu_file};
menu_usb_load=if test "x${mfg_usb_fstype}" = "xext2"; then run menu_usb_load_ext2 ; else ; run menu_usb_load_fat ; fi
menu_nodhcp=run clear_filesize ; run menu_load ; if test 0${filesize} -gt 0; then autoscr ${menu_addr_r}; else ; echo Failed menu load ; fi
menu=dhcp ; run clear_filesize ; run menu_load ; if test 0${filesize} -gt 0; then autoscr ${menu_addr_r}; else ; echo Failed menu load ; fi
fw_file=u-boot.bin
fw_load=tftp ${fw_addr_r} ${mfg_root}${mfg_dir}/${fw_file}; if test $? -ne 0; then run clear_filesize ; echo Download failed ;fi
fw_usb_load_ext2=usb start; ext2load usb ${mfg_usb_dev}:${mfg_usb_part} ${fw_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${fw_file};
fw_usb_load_fat=usb start; fatload usb ${mfg_usb_dev}:${mfg_usb_part} ${fw_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${fw_file};
fw_usb_load=if test "x${mfg_usb_fstype}" = "xext2"; then run fw_usb_load_ext2 ; else ; run fw_usb_load_fat ; fi
fw_update_raw=protect off 0xFFFA0000 FFFFFFFF;erase 0xFFFA0000 FFFFFFFF;cp.b ${fw_addr_r} 0xFFFA0000 ${filesize};cmp.b ${fw_addr_r} 0xFFFA0000 ${filesize};setenv filesize; saveenv
fw_usb_update=run clear_filesize ; run fw_usb_load ; if test 0${filesize} -gt 0; then run fw_update_raw ; else ; echo Failed update load ; fi
fw_update_nodhcp=run clear_filesize ; run fw_load ; if test 0${filesize} -gt 0; then run fw_update_raw ; else ; echo Failed update load ; fi
fw_update=dhcp ; run clear_filesize ; run fw_load ; if test 0${filesize} -gt 0; then run fw_update_raw ; else ; echo Failed update load ; fi
boot_common_args=run addtty addmisc
mfg_usb_dir=mlnx460ex
mfg_usb_load_ext2=usb start; echo "Loading ${mfg_kernel_file}";ext2load usb ${mfg_usb_dev}:${mfg_usb_part} ${kernel_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${mfg_kernel_file};echo "Loading ${mfg_fdt_file}"; ext2load usb ${mfg_usb_dev}:${mfg_usb_part} ${fdt_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${mfg_fdt_file};echo "Loading ${mfg_ramdisk_file}"; ext2load usb ${mfg_usb_dev}:${mfg_usb_part} ${ramdisk_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${mfg_ramdisk_file}
mfg_usb_load_fat=usb start; echo "Loading ${mfg_kernel_file}";fatload usb ${mfg_usb_dev}:${mfg_usb_part} ${kernel_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${mfg_kernel_file};echo "Loading ${mfg_fdt_file}"; fatload usb ${mfg_usb_dev}:${mfg_usb_part} ${fdt_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${mfg_fdt_file};echo "Loading ${mfg_ramdisk_file}"; fatload usb ${mfg_usb_dev}:${mfg_usb_part} ${ramdisk_addr_r} ${mfg_usb_root}${mfg_usb_dir}/${mfg_ramdisk_file}
mfg_usb_load=if test "x${mfg_usb_fstype}" = "xext2"; then run mfg_usb_load_ext2 ; else ; run mfg_usb_load_fat ; fi
mfg_usb=echo "Manufacture will load from USB directory ${mfg_usb_root}${mfg_usb_dir}, and boot"; echo; run clear_filesize ; run mfg_usb_load; if test 0${filesize} -gt 0; then echo Booting mfg ; run mfg_args mfg_common_args;bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} ; else ; echo Failed mfg load ; fi
kernel_addr=ff000000
fdt_addr=ff1e0000
ramdisk_addr=ff200000
fw_addr_r=400000
menu_addr_r=B00000
pciconfighost=1
pcie_mode=RP:RP
autoload=no
boot_usb_ext2_loc_1=run usb_args_loc_1 boot_common_args;echo "Loading ${boot_kernel_file}";ext2load usb ${boot_usb_dev}:${boot_usb_part_loc_1} ${kernel_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_kernel_file};echo "Loading ${boot_fdt_file}";ext2load usb ${boot_usb_dev}:${boot_usb_part_loc_1} ${fdt_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_fdt_file};bootm ${kernel_addr_r} - ${fdt_addr_r}
boot_usb_ext2_loc_2=run usb_args_loc_2 boot_common_args;echo "Loading ${boot_kernel_file}";ext2load usb ${boot_usb_dev}:${boot_usb_part_loc_2} ${kernel_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_kernel_file};echo "Loading ${boot_fdt_file}";ext2load usb ${boot_usb_dev}:${boot_usb_part_loc_2} ${fdt_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_fdt_file};bootm ${kernel_addr_r} - ${fdt_addr_r}
boot_usb_fat_loc_1=run usb_args_loc_1 boot_common_args;echo "Loading ${boot_kernel_file}";fatload usb ${boot_usb_dev}:${boot_usb_part_loc_1} ${kernel_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_kernel_file};echo "Loading ${boot_fdt_file}";fatload usb ${boot_usb_dev}:${boot_usb_part_loc_1} ${fdt_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_fdt_file};bootm ${kernel_addr_r} - ${fdt_addr_r}
boot_usb_fat_loc_2=run usb_args_loc_2 boot_common_args;echo "Loading ${boot_kernel_file}";fatload usb ${boot_usb_dev}:${boot_usb_part_loc_2} ${kernel_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_kernel_file};echo "Loading ${boot_fdt_file}";fatload usb ${boot_usb_dev}:${boot_usb_part_loc_2} ${fdt_addr_r} ${boot_usb_root}${boot_usb_dir}/${boot_fdt_file};bootm ${kernel_addr_r} - ${fdt_addr_r}
mfg_kernel_file=vmlinuz
mfg_ramdisk_file=rootfs
mfg_ramdisk_size=180224
mfg_fdt_file=fdt
mfg_usb_dev=0
mfg_usb_part=1
mfg_usb_fstype=fat
mfg_usb_root=/
boot_kernel_file=vmlinuz
boot_fdt_file=fdt
boot_usb_dev=0
boot_usb_part_loc_1=2
boot_usb_part_loc_2=3
boot_usb_root_loc_1=/dev/sda5
boot_usb_root_loc_2=/dev/sda6
usb_args_loc_1=setenv bootargs root=${boot_usb_root_loc_1} ro reset_button=${reset_button} rootdelay=8 ${image_kernel_args} ${extra_args}
usb_args_loc_2=setenv bootargs root=${boot_usb_root_loc_2} ro reset_button=${reset_button} rootdelay=8 ${image_kernel_args} ${extra_args}
jffs2_args=setenv bootargs root=${rootdev} rootfstype=jffs2 ro reset_button=${reset_button} ${image_kernel_args} ${extra_args}
mfg_extra_args=ramdisk=65536
ethaddr=xx:xx:xx:xx:xx:xx
eth1addr=yy:yy:yy:yy:yy:yy
ethact=ppc_4xx_eth0
bootfile=mlnx405ex/uImage
script_rev=3.3.0
script_date=7.12.13
bootdelay=5
emcram_addr=400000
emcfl_one_start=ff400000
emcfl_one_end=ff5fffff
emcfl_two_start=ff600000
emcfl_two_end=ff7fffff
emcload_addr=ff600000
binary_name=ibsw.bin.load
restore_defaults=setenv load_dir .; setenv binary_name ibsw.bin.load; saveenv;
setip_176=setenv ipaddr 192.168.176.190; setenv serverip 192.168.176.253; saveenv
setip_177=setenv ipaddr 192.168.177.191; setenv serverip 192.168.177.253; saveenv
setip_linux=setenv ipaddr 192.168.10.10; setenv serverip 192.168.10.1; saveenv
setip_16=setenv ipaddr 172.16.255.120; setenv serverip 172.16.255.252; saveenv
setip_17=setenv ipaddr 172.17.255.120; setenv serverip 172.17.255.252; saveenv
set_fab_a=setenv emc_fabric A; saveenv
set_fab_b=setenv emc_fabric B; saveenv
emcpxe=tftp ${emcram_addr} ${load_dir}/${binary_name}
emcflash=ping $serverip; bootm ${emcload_addr}
mlxlinux=run jffs2_args boot_common_args;bootm ${kernel_addr} - ${fdt_addr}
flash_jffs2=run emcflash
menu_usb=run emcflash
emcburn_one=prot off ${emcfl_one_start} ${emcfl_one_end}; erase ${emcfl_one_start} ${emcfl_one_end}; setenv autostart no; tftp ${emcram_addr} ${load_dir}/${binary_name}; iminfo ${emcram_addr}; cp.b ${emcram_addr} ${emcfl_one_start} ${filesize}; iminfo ${emcfl_one_start}; setenv autostart yes; prot on ${emcfl_one_start} ${emcfl_one_end};
emcburn_two=prot off ${emcfl_two_start} ${emcfl_two_end}; erase ${emcfl_two_start} ${emcfl_two_end}; setenv autostart no; tftp ${emcram_addr} ${load_dir}/${binary_name}; iminfo ${emcram_addr}; cp.b ${emcram_addr} ${emcfl_two_start} ${filesize}; iminfo ${emcfl_two_start}; setenv autostart yes; prot on ${emcfl_two_start} ${emcfl_two_end};
emcload_one=setenv emcload_addr ${emcfl_one_start}; saveenv
emcload_two=setenv emcload_addr ${emcfl_two_start}; saveenv
script_get=setenv autostart no; tftp ${emcram_addr} uboot_start_script.img; setenv autostart yes; imi ${emcram_addr};
script_exe=autoscr ${emcram_addr}
boot_emcpxe=setenv flash_jffs2 run emcpxe; setenv menu_usb run emcpxe; saveenv
boot_emcflash=setenv flash_jffs2 run emcflash; setenv menu_usb run emcflash; saveenv
boot_mlxlinux=setenv flash_jffs2 run mlxlinux; setenv menu_usb run mlxlinux; saveenv
emchelp=echo All commands are preceeded by \"run \"; echo setip_176 ...... set ip and server ip to 192.176 subnet; echo setip_177 ...... set ip and server ip to 192.177 subnet; echo setip_16 ....... set ip and server ip to 172.16 subnet; echo setip_17 ....... set ip and server ip to 172.17 subnet;echo setip_linux .... set ip and server ip to linux default (10.10); echo emcburn_one .... burn ibsw.bin.load to the first symmk flash location;echo emcburn_two .... burn ibsw.bin.load to the second symmk flash location;echo emcload_one .... set emcflash to point to the first symmk flash location;echo emcload_two .... set emcflash to point to the second symmk flash location;echo script_get ..... Load a new version of the uboot_start_script.img;echo script_exe ..... Execute the script loaded by script_get (check CRC first!);echo script_status .. Dump revision and build date of last run script; echo boot_emcpxe .... Set PXE boot to be the default boot;echo emcpxe ......... Run PXE boot (just this time); echo boot_emcflash .. Set the Symmk in flash to be the default boot;echo emcflash ....... Boot (just this time) from flash;echo boot_mlxlinux .. Set the MLX linux version to be the default boot;echo mlxlinux ....... Run MLX linux (just this time);echo set_fab_a ...... Set switch to be fabric A; echo set_fab_b ...... Set switch to be fabric B
script_status=echo Last Script Executed; echo Script Revision: 3.3.0; echo Script Built on: 7.12.13;
load_dir=os/EMC/OS-EMC_1.383_IB-PPC_M460EX
filesize=1A04F0
fileaddr=400000
gatewayip=10.10.4.1
serverip=192.168.138.101
dnsip=192.168.138.101
emc_fabric=B
autostart=off
autoload=off
ipaddr=192.168.1.10
netmask=255.255.255.0
rootdev=/dev/mtdblock7
image_kernel_args=loglevel=2
stdin=serial
stdout=serial
stderr=serial
reset_button=0
ver=U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82-EMC ppc (Feb 27 2013 - 12:13:42)

Environment size: 12586/16380 bytes
=>
 

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
Thanks a bunch; (un?)fortunately I'm not missing anything, but it's good to be sure.

That said, oh my god, I'm an idiot. Remember a few posts back where I showed how to flash from within the CLI fae menu?

This line is wrong:
Code:
sx6012 [standalone: master] (fae) # flint -allow_psid_change -override_cache_replacement -d /dev/mst/mt51000_pci_cr0 -i /tmp/user.md5sum b
Correct device path (for mine at least) is /dev/mst/mt51000_pciconf0. Reflashed, reinstalled, and it works now. Yay!
 

klui

Active Member
Feb 3, 2019
160
67
28
Good to know you got it fixed and informed the thread. Was it force of habit of using pci_cr0 for their NIC cards? mst status doesn't show that device. Only dev-i2c-[0-9] and mt51000_pciconf0.
 

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
I think I saw it listed under one of the other status commands... *sigh*

It's still very unhappy about the fans all being dead, but it's not generating 10MB of syslog messages every few minutes, so that's a plus.
It also still won't recognize any SFP+ modules installed in a QSA... but it does identify the Mellanox Infiniband DAC

I also only get two i2c devices on initial boot;

Code:
MST modules:
------------
    MST PCI module loaded
    MST PCI configuration module loaded

MST devices:
------------
/dev/mst/dev-i2c-0         - Embedded I2C master
/dev/mst/dev-i2c-1         - Embedded I2C master
/dev/mst/mt51000_pciconf0        - PCI configuration cycles access.
                                   domain:bus:dev.fn=0001:81:00.0 addr.reg=88 data.reg=92
                                   Chip revision is: 02
I get all of them once I do an 'mst restart'... tc and hwd are also pretty upset at my PSU/fan situation;

Code:
May 23 21:55:33 sx6012 temp_control[5003]: [tc.NOTICE]: Only:[0] fans out of :[4] are working,TC is setting All Chassis fans to:[99%],(Maximal speed)
May 23 21:55:33 sx6012 temp_control[5003]: [tc.NOTICE]: Could not set fan:[/MGMT/FAN1/f1] speed, because it's not working or not present
May 23 21:55:33 sx6012 temp_control[5003]: [tc.NOTICE]: Could not set fan:[/MGMT/FAN2/f1] speed, because it's not working or not present
May 23 21:55:33 sx6012 temp_control[5003]: [tc.NOTICE]: Could not set fan:[/MGMT/FAN3/f1] speed, because it's not working or not present
May 23 21:55:33 sx6012 temp_control[5003]: [tc.NOTICE]: Could not set fan:[/MGMT/FAN4/f1] speed, because it's not working or not present
May 23 21:55:49 sx6012 hwd[5055]: TID 1208143072: [hwd.WARNING]: adjust_pses_number_according_to_machine: P/N:[(null)] for Dingo is unknown. Treating it as 1PSU Dingo
May 23 21:56:20 sx6012 hwd[5055]: TID 1208143072: [hwd.WARNING]: adjust_pses_number_according_to_machine: P/N:[(null)] for Dingo is unknown. Treating it as 1PSU Dingo
These messages repeat over and over; at this point it's probably fairly safe to say this is a hardware problem, though.
 

klui

Active Member
Feb 3, 2019
160
67
28
At least the system doesn't shut down.

You may want to consider redirecting all that to a remote server otherwise it will just wear out your flash.
 

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
Yeaaaah, it's pretty much toast. Something's screwy with the i2c buses, which are on the SwitchX chip and/or the Lattice CPLD, and there's nothing obvious I can find other than a couple of blown zero-ohm resistors that shouldn't be relevant (and replacing didn't fix)

Ports 1-4 don't work, ports 5-6 do, 7-8 don't, 9-12 do.

Amusingly, putting an optic in port 4 makes it report one of the fans is working at 18,000rpm; these are 3000rpm fans :p

Replacement time! But hey, now I know exactly what to do to the next one.
 

anrusakov

New Member
Jan 4, 2020
7
0
1
Hi,

Greate post, thank for every one involved.

I am migrating reflashing 2 6012.
1st one is - OK - i was able to reflash it (it take some time, but it is working)

But, i am stuck on 2nd.
I try 1002 and 8012, everething ok till 6.2.b, i am booting to MLNX-OS, but admin/admin didn't work.


removing and/or modifing /mnt/root2/etc/passwd didn't help.
 

klui

Active Member
Feb 3, 2019
160
67
28
everething ok till 6.2.b, i am booting to MLNX-OS, but admin/admin didn't work.

removing and/or modifing /mnt/root2/etc/passwd didn't help.
Post output from

ls -l /mnt/root2/etc/passwd /mnt/root2/var/opt/tms/output/shadow
cat /mnt/root2/etc/passwd
cat /mnt/root2/var/opt/tms/output/shadow
 

anrusakov

New Member
Jan 4, 2020
7
0
1
klui
I was able to find solution.

I gor 2 switches ...

And seems 1st is ok to upgrade using standart guid.
2nd work only with chrood upgrade. When I remove/change link to passwd file - i can't login.

So seems like
Step 6.2.C
keep passwd and shadow as is

Upgrade system using chroot method from #372
reboot, login using admin/admin


And was able to upgrade to 1002 and 8012.

But still got a question.
The Plan is to use switches as IB FDR (56G).
I got several DAC cables some of them got MAX speed info
1591457653097.png

But some of them not

Infiniband speeds : unknown

The problem is that i got 40G QDR all the time - on clevre and stupid cables

Does any one able to use them as 56Gb IB with non original Cables?


P.S. I got 56G Cards and 2nd switch :)
 

Eong

Member
Dec 24, 2019
35
7
8
I'm kind of surprised that nobody's discovered this hidden menu yet, but here's a fix for that;
Code:
configure terminal
fae cable-stamping-unlock 40g_lr4
exit
write memory
Not 100% sure I remember that right, but fae cable-stamping-unlock ? will show you the available unlocks - there's one other, which IIRC allows using 56Gb rates over 40Gb-rated DACs.

The fae menu has an awful lot of fun commands hidden inside; It doesn't show up in a regular ? or help listing, but if you type fae ? while in configure mode, it'll list out (most of) the available commands. I believe flint is included - not sure, my switch is re-imaging itself after I messed up the latest conversion attempt - along with a number of other executables in the base linux OS and a whole bunch of fun debug/manufacturing commands.

Be careful. Several of the commands in here can near-irrevocably break your switch.
I was too busy in the past few weeks with my project and I didn't visit forum.
I also found that and it fixed my issue. All the LR4 transceivers I have work without any problem.
Thanks for the tips anyway.
Did you figure out how to remove the minimal requirement of the fan speed?
 
Last edited:

neggles

is 34 Xeons too many?
Sep 2, 2017
48
15
8
Melbourne, AU
omnom.net
klui
I got several DAC cables some of them got MAX speed info
View attachment 14420

But some of them not

Infiniband speeds : unknown

The problem is that i got 40G QDR all the time - on clevre and stupid cables

Does any one able to use them as 56Gb IB with non original Cables?


P.S. I got 56G Cards and 2nd switch :)
I can't remember the specific command, but fae cable-stamping-unlock ? at the configuration prompt should give you a list of two options. One is 40g_lr4, the name of the other one implied that it would unlock 56G over unsupported cables.
 

anrusakov

New Member
Jan 4, 2020
7
0
1
I can't remember the specific command, but fae cable-stamping-unlock ? at the configuration prompt should give you a list of two options. One is 40g_lr4, the name of the other one implied that it would unlock 56G over unsupported cables.
Yep, already done - But the only option is to enable fdr10.
And after that i got fdr10, but still - 40G

1591688112233.png

I got same problem on 8012, but seems i am not able to get correct GID :( - as license is empty.
Now i am on 1002 - license is ok, but still no Full FDR

P.S. regurding 8012 - what is correct way to get 12 digit hex for hostid GUID from switch label didn't work (as it 16 digits)
 

Eong

Member
Dec 24, 2019
35
7
8
Yep, already done - But the only option is to enable fdr10.
And after that i got fdr10, but still - 40G

View attachment 14457

I got same problem on 8012, but seems i am not able to get correct GID :( - as license is empty.
Now i am on 1002 - license is ok, but still no Full FDR

P.S. regurding 8012 - what is correct way to get 12 digit hex for hostid GUID from switch label didn't work (as it 16 digits)
Any digits will work. Like 010101010101.
 

anrusakov

New Member
Jan 4, 2020
7
0
1
I think it means that non mellanox qsfp14 cables can be used with that switch.
Nope, there is only fdr10, so you can't unlock 56 FDR.

fae cable-stamping-unlock ?
<type> Cable or transceiver speed
fdr10
40g_lr4
 

up3up4

Member
Jun 10, 2018
44
10
8
Nope, there is only fdr10, so you can't unlock 56 FDR.

fae cable-stamping-unlock ?
<type> Cable or transceiver speed
fdr10
40g_lr4
Nope, there is only fdr10, so you can't unlock 56 FDR.

fae cable-stamping-unlock ?
<type> Cable or transceiver speed
fdr10
40g_lr4
You need a cable or optical transceiver physically support FDR 14/56G speed, even you have a license actived the switch 56G speed.
 

anrusakov

New Member
Jan 4, 2020
7
0
1
You need a cable or optical transceiver physically support FDR 14/56G speed, even you have a license actived the switch 56G speed.
So, yes i do understand ... But.
I do understand why clever QDR cables didn't work with FRD10, but i don't understand why stupid cables can work with FRD10.

So basicly the major question - Does anyone have stupid QSFP cable with FDR/56G ?