Not at all. You can add multipathing the same way as for iSCSI. Have a look at VMware documentation for iSER and you'll see all these details there.
Beware of EMC switches sold as Mellanox SX6XXX on eBay
- Thread starter mpogr
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Well this would be the configuration on an active/active cluster, but unlike in my planed active/passive configuration the failover host is not representing any iSCSI/iSER LUNs while primary host is alive, so ESXi is not using multipath to two individual targets on two physical boxes, multipath is only be used in case of switch/nic death (but still connected to the same physical box, only using the failover network path (second nic/second switch)). In my iSCSI setup in case of primary failure the secondary host would import the ZFS pools, start the SCST target and then taking over the floating cluster IP. Am I thinking to much/wrong here? 
For an active/active solution I would have to take DRDB, but because of performance reason I dislike that. An active/passive solution with a "re-uptime" of a couple of seconds is enough in my case.
Some guy wrote a guide with a very similar setup. But instead of iSCSI and a ZFS on dual-ported SAS disks I want to use iSER and a ZFS on dual-ported NVMe disks.
https://bm-stor.com/index.php/blog/Linux-cluster-with-ZFS-on-Cluster-in-a-Box/
For an active/active solution I would have to take DRDB, but because of performance reason I dislike that. An active/passive solution with a "re-uptime" of a couple of seconds is enough in my case.
Some guy wrote a guide with a very similar setup. But instead of iSCSI and a ZFS on dual-ported SAS disks I want to use iSER and a ZFS on dual-ported NVMe disks.
https://bm-stor.com/index.php/blog/Linux-cluster-with-ZFS-on-Cluster-in-a-Box/
Last edited:
today chat with one friend, he bricked his sx6012 after execute:
eeprom read 0x53 0x8500000 0 1958000
in uboot ，the switch report i2c error and freeze. he just unplug power cable
after that the switch just dead can’t boot up,is there any chance rescue his sx6012?
eeprom read 0x53 0x8500000 0 1958000
in uboot ，the switch report i2c error and freeze. he just unplug power cable
after that the switch just dead can’t boot up,is there any chance rescue his sx6012?
Last edited:
Hello,
I am looking to buy a Mellanox SX6012 but have some questions after reading the forum and other documentation on the net.
There are 2 models for SX6012 the "T" version which is FDR10 and "F" version which is FDR.
Q1: Is there any hardware difference between them? Or is just software/license?
Now about OEM ones:
EMC models:
100-886-236-00
100-886-236-04
Q2: What's the difference between them? One is FDR10 and one is FDR?
HP Models:
They don't have the SX6012 it seems, only the 36 ports or 216 ports models.
Q3: Is there any HP SX6012? If yes what is the HP part number?
IBM and Lenovo Models:
The same like HP they don't seem to have any SX6012 but they have the SX6036 models.
Q4: Is there any IBM\Lenovo SX6012? If yes what is the IBM/Lenovo part number?
Dell EMC Models:
It seems on Mellanox site there is no mention about this SX6012 from EMC.
All Dell EMC switches in the list seems to be the externally managed ones and there is no internally managed ones.
Q5: Is this correct?
U-Boot and firmware questions:
I would to know the status of the U-Boot for standard and OEM models.
It seems only the EMC model has the U-Boot without password. In rest all have the U-Boot locked.
Q6: Can someone comment on this? Are there any other models from OEM or not with U-Boot unlocked (without password)?
Firmware for Mellanox switches can be in 2 categories from what I read:
A) externally managed firmware, usually very small image around 1.5MB or less with image file name for example "fw-sx-9_4_2000-Ax.bin"
B) internally managed firmware, more exactly the Mellanox OS. These images are 200~500MB and have the file naming for example "image-PPC_M460EX-3.5.1006.img"
Q7: Are the "image-PPC_M460EX-3.y.abcd.img" files interchangeable between all internally managed switches like 12/18/36 ports?
Q8: Are the "image-PPC_M460EX-3.y.abcd.img" files customized for OEMs or the Mellanox and OEM files are identical?
Q9: Can U-Boot be unlocked if it locked? Can it be flashed with an unlocked U-Boot image?
And last part for now regarding access to Linux shell for internally managed switches.
Q10: How can you drop from normal switch CLI to Linux shell? I saw people posting different commands run from Linux shell.
I saw during booting output that switch is reporting 2 storage parts a FLASH: 16 MB and a NAND: 1024 MiB.
I suppose the 16MB is for the U-Boot and the 1024MB is for the "image-PPC_M460EX-3.y.abcd.img" which should be the Mellanox OS.
Q11: Can someone confirm this?
Regards!
I am looking to buy a Mellanox SX6012 but have some questions after reading the forum and other documentation on the net.
There are 2 models for SX6012 the "T" version which is FDR10 and "F" version which is FDR.
Q1: Is there any hardware difference between them? Or is just software/license?
Now about OEM ones:
EMC models:
100-886-236-00
100-886-236-04
Q2: What's the difference between them? One is FDR10 and one is FDR?
HP Models:
They don't have the SX6012 it seems, only the 36 ports or 216 ports models.
Q3: Is there any HP SX6012? If yes what is the HP part number?
IBM and Lenovo Models:
The same like HP they don't seem to have any SX6012 but they have the SX6036 models.
Q4: Is there any IBM\Lenovo SX6012? If yes what is the IBM/Lenovo part number?
Dell EMC Models:
It seems on Mellanox site there is no mention about this SX6012 from EMC.
All Dell EMC switches in the list seems to be the externally managed ones and there is no internally managed ones.
Q5: Is this correct?
U-Boot and firmware questions:
I would to know the status of the U-Boot for standard and OEM models.
It seems only the EMC model has the U-Boot without password. In rest all have the U-Boot locked.
Q6: Can someone comment on this? Are there any other models from OEM or not with U-Boot unlocked (without password)?
Firmware for Mellanox switches can be in 2 categories from what I read:
A) externally managed firmware, usually very small image around 1.5MB or less with image file name for example "fw-sx-9_4_2000-Ax.bin"
B) internally managed firmware, more exactly the Mellanox OS. These images are 200~500MB and have the file naming for example "image-PPC_M460EX-3.5.1006.img"
Q7: Are the "image-PPC_M460EX-3.y.abcd.img" files interchangeable between all internally managed switches like 12/18/36 ports?
Q8: Are the "image-PPC_M460EX-3.y.abcd.img" files customized for OEMs or the Mellanox and OEM files are identical?
Q9: Can U-Boot be unlocked if it locked? Can it be flashed with an unlocked U-Boot image?
And last part for now regarding access to Linux shell for internally managed switches.
Q10: How can you drop from normal switch CLI to Linux shell? I saw people posting different commands run from Linux shell.
I saw during booting output that switch is reporting 2 storage parts a FLASH: 16 MB and a NAND: 1024 MiB.
I suppose the 16MB is for the U-Boot and the 1024MB is for the "image-PPC_M460EX-3.y.abcd.img" which should be the Mellanox OS.
Q11: Can someone confirm this?
Regards!
Finaly my switches and cards arrived, but no cables
How can I find out the switchlid?
How can I find out the switchlid?
Code:
mst status
MST modules:
------------
MST PCI module loaded
MST PCI configuration module loaded
MST devices:
------------
/dev/mst/dev-i2c-0 - Embedded I2C master
/dev/mst/dev-i2c-1 - Embedded I2C master
/dev/mst/mt51000_pciconf0 - PCI configuration cycles access.
domain:bus:dev.fn=0001:81:00.0 addr.reg=88 data.reg=92
Chip revision is: 02I can't remember the detailed process but if you need to flash the switch from a linux box then you don't need it, see https://forums.servethehome.com/ind...anox-sx6xxx-on-ebay.10786/page-10#post-206932 for switch - local flashing instructions
Another success story here. Thanks to everyone sharing information!
I had to change the default boot env for uboot, havent found that in the guide, for anyone looking for it:
I had to change the default boot env for uboot, havent found that in the guide, for anyone looking for it:
Code:
setenv rootdev /dev/mtdblock7
setenv bootcmd "run mlxlinux"
saveenvHi everyone! I've read through this thread and learned much by pieces of wisdom from Mr. mpogr and all your replies. Cheers!
I've PMed Mr. mpogr and got a conversion guide, but with some attachments missing. I contacted him again when I finally received the switch, and it seems he's not online these days...
I cannot wait to convert it so can anyone kindly send me the missing pieces instead? Things I need are "/etc/image_layout.sh" "chad" and "hwd".
Thanks in advance!
I've PMed Mr. mpogr and got a conversion guide, but with some attachments missing. I contacted him again when I finally received the switch, and it seems he's not online these days...
I cannot wait to convert it so can anyone kindly send me the missing pieces instead? Things I need are "/etc/image_layout.sh" "chad" and "hwd".
Thanks in advance!
OK. Finally I found another way, without touching the shipped "chad" or "hwd". It is even more "hacking" so I'll not publish it anywere...
Now working on "image_layout.sh". Seems it should be rather simple (< 10 lines or so), but more investigation should be made for sure.
Now working on "image_layout.sh". Seems it should be rather simple (< 10 lines or so), but more investigation should be made for sure.
Sorry, I don't have a solution but I'm interested to know why he did that? There are no devices on slave address 0x53, unless I'm mistaken.
If anyone is tempted to play with the i2c commands in uboot I'd strongly advise against it. I bricked mine by reading too but managed to bring it back to life with a Diolan DLN-2-U2C via one of the internal i2c headers. I think i2c support in that uboot build is broken.
Edit: for reference, this is the command that bricked mine:
imd 0x52 0.2 0xFFF
That is supposed to read but actually wrote a zero to the first byte of the cpu bootstrap code in the boot rom
Last edited:
Hello everyone,
Reading through the guide and am looking at the contents of MLNX-OS image (file name image-PPC_M460EX-ppc-m460ex-20160609-202426.tgz) and I can't seem to find an 'image_layout.sh' in that image. I could locate the other file chad, hwd and ibd in the image that should the guide states should be replaced.
If I attempt the conversion, do I just copy image_layout.sh to /mnt/root2/etc even though it does not exist?
Please advise if it's ok for the file to no exist in the original image or if I got a wrong image.
Thanks
Reading through the guide and am looking at the contents of MLNX-OS image (file name image-PPC_M460EX-ppc-m460ex-20160609-202426.tgz) and I can't seem to find an 'image_layout.sh' in that image. I could locate the other file chad, hwd and ibd in the image that should the guide states should be replaced.
If I attempt the conversion, do I just copy image_layout.sh to /mnt/root2/etc even though it does not exist?
Please advise if it's ok for the file to no exist in the original image or if I got a wrong image.
Thanks
Last edited:
