40Gbe+ TOR Switch Recommendations

[Toddh]

I’m looking for recommendations on 40GbE QSFP+ or faster switches.

It will be used for S2D Clusters so needs to be fully managed and support DCB, PFC and RDMA. I would prefer something with a GUI, or at least a CLI menu (like the HP switches), 16 – 36 Ports

Used reliable enterprise sub $1000 each.

I am currently using Mellanox MCX354-FCBT cards for our 56Gb Infiniband and have spares to be used for the Ethernet. I was hoping to find a Mellanox switch but there seem to be no cheap MSX1036s. Been using the Mellanox Infiniband switches for years so I am familiar with their GUI.

I started reading on the Celestica switches but it seems the ones on eBay you have to install a Linux OS to get a GUI and I am not sure how difficult a task that is.

Frankly there’s just so much information here on the forum that I was hoping those of you who have gone this direction could help narrow it down.

Todd

 

[fohdeesha]

your best bet here is probably an arista 7050QX-32s, I would abandon the requirement of a GUI on a switch like this at your requested price point, at least a GUI worth using. The Celecticas you mention have a known failure problem as they came with the intel CPUs that had a manufacturing defect, one of the clock lines deteriorates until it will no longer boot. Both Celestica D4040s I've bought off ebay failed in this manner

 

[Terry Wallace]

Why not buy an sx6036
Mellanox 36 Port QSFP FDR Managed InfiniBand Switch SX6036 | eBay

 

[BeTeP]

Both Celestica D4040s I've bought off ebay failed in this manner

I presume you were able to fix them weren't you? Using @okrasit's method?

 

[fohdeesha]

I presume you were able to fix them weren't you? Using @okrasit's method?

one I was able to solder in the resistor patch indeed, the other one, even though being an identical D4040, had slightly different test pad breakouts and didn't have the clock line pad that he outlined (and that my other D4040 did). Probed around the board for about 30 minutes looking for another place to grab the LPC clock before giving up and moving on, in my mind any switch important enough to go in a DC and have production traffic running through it probably shouldn't require home soldering to stay alive

 

[Toddh]

What about the Celestica DX010, supposing I can locate a couple? Is there an OS that can be loaded with a GUI?

We are using an Arista 7050QX-32 now. Nothing wrong with the Arista. It is running an older firmware I have not taken the time to learn EOS with everything else I have going on so when I want to check status of the switch or network I am at a loss. Which is why I was leaning towards finding something with a GUI.


.

 

[Toddh]

Correct me if I am wrong, but the Mellanox SX6036 is an IB switch. It can run Ethernet but you need a rather expensive license.

We have one now but the Ethernet section of the GUI is greyed out.

Admittedly I have never tried to connect a QSFP to it as Ethernet.

 

[i386]

What about the Celestica DX010, supposing I can locate a couple? Is there an OS that can be loaded with a GUI?

We are using an Arista 7050QX-32 now. Nothing wrong with the Arista. It is running an older firmware I have not taken the time to learn EOS with everything else I have going on so when I want to check status of the switch or network I am at a loss. Which is why I was leaning towards finding something with a GUI.

GUIs are usually features of soho/smb switches. Like @fohdeesha said it's unlikely to find a switch os with a gui and your requirements.

I would say bite the bullet and learn how to use the commands in cli/eos, it's very similar to ios and can help in future jobs/to get more into networking

 

[Rand__]

What about the Celestica DX010, supposing I can locate a couple? Is there an OS that can be loaded with a GUI?.

SoNIC does not have a GUI and I dont think you want to spend another 2K on any of the commercial NOSs running on it even if they had a GUI...