Beware of EMC switches sold as Mellanox SX6XXX on eBay

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
I am in vpi mode and I just enabled sm manager
Weird... I tried it a few months ago and got different error messages when I tried to enable the sm manager with the vpi profile.

Will try this weekend again and report back.
 

mpogr

Active Member
Jul 14, 2016
115
95
28
53
The short answer is no, at least not easily. PM me for the longer answer...
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Got mine in today. Holy s*** this thing is DEEP! Anyone got a line on the rack mounting kit?
 

Hindsight

Member
Mar 28, 2016
55
14
8
42

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
I got to the point where I need to flash the switch, but now I'm stuck. Is the switch picky about optics? I can see the red light in the QSFP on my card, but none of the ones I plug into the switch light up. I tried running opensm, but it just keeps saying SM port is down.
 

Hindsight

Member
Mar 28, 2016
55
14
8
42
The greatest thing about the Mellanox nics and switches is it doesn't really care what type of transceivers you use.
run opensm on your pc and then make sure the switch is back into it's regular mode (not linux) to flash.
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Fully booted the upgraded firmware, but still no link or light in the fiber. The only QSFPs I have are the Brocade XBR-000232s and Cisco QSFP-40G-SR-BD.
 

Hindsight

Member
Mar 28, 2016
55
14
8
42
Can you explain what you mean by upgraded firmware? You were able to upgrade the switch through IB? After that unless you are running opensm it won't be on but you don't need it to be. Once you get the full os booted they will be in vpi mode and come online and you can check the port status from there.
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Looking in the fiber, I see the ConnectX3 has red light, but nothing on the switch side.
 

mpogr

Active Member
Jul 14, 2016
115
95
28
53
As @Hindsight pointed out, you need to run opensm first. Make sure it's configured properly. The lights will come on then.
If anything else fails, you can also run flint on the switch itself. PM me for details.
 

MikeWebb

Member
Jan 28, 2018
92
29
18
53
I might have a play around with this on my 6012. Looks like a good project, well done. PM sent
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Finally got it working after ordering a DAC. Not sure what the issue was since now my other optics seem to work fine in ETH and IB mode.

Quick tip, since I went through the whole procedure a few times trying to figure out what the issue was.
Instead of copying the image.tgz into /mnt/var, do mount -t tmpfs tmpfs /mnt/tmp and use it instead. Saves a lot of time and some writes to the flash.
 
  • Like
Reactions: metag

Hindsight

Member
Mar 28, 2016
55
14
8
42
Least it's working. I went all DAC for my servers and only using fiber to my desktop and other 10gb LCs with the mtp - 4x LC breakout cable. DACs are nice and cheap.
 

mb300sd

Active Member
Aug 1, 2016
204
80
28
34
Is there any way to set up inband management on the ethernet side? I can't seem to find any options to do so. If not I guess I'll just connect mgmt2 to my other switch.
 

Hindsight

Member
Mar 28, 2016
55
14
8
42
Is there any way to set up inband management on the ethernet side? I can't seem to find any options to do so. If not I guess I'll just connect mgmt2 to my other switch.
http://www.mellanox.com/related-docs/prod_management_software/MLNX-OS_ETH_v3_6_3508_UM.pdf

Mellanox switches support out-of-band (OOB) dedicated interfaces (e.g. mgmt0, mgmt1) and in-band dedicated interfaces.

Scroll to 4.1.5 In-Band Management

Edit - Some of that was blank, This may be better.

HowTo Enable Inband Management on Mellanox Ethe... | Mellanox Interconnect Community