Cheapest, quietest, lowest power 40Gbe switch? (and preferably less than 55cm deep)

solon

Member
Apr 1, 2021
44
2
8
I've recently set up a 40Gb SAN at home as Covid keep oneself occupied therapy, but I've come to the conclusion that I als want to run parallel 40GBE. I have Mellanox MCX354A-QCBT cards which I'll flash to FCBT, ideally using one port for the SAN and the other for 40Gbe. I have a mellanox IS5022 modded with a 200mm noctua fan (those bastards use a non-standard 3 pin fan connector by the way, which fried a cheap test fan) and a little 3D printed air guide, which works fine for the SAN and pulls about 20W.

I stil have plenty of spare QDR cables, so as far as I can tell all I really need is a 40Gbe switch.

So, I'm wondering what people's thoughts are on the best way to go with this. I've been looking at the mellanox SX6012 or SX6015 switches which seem like they'll work and can probably be made silent with a small fortune in Noctua fans.

I'm not all that enthusiastic about how much power (spec states a ludicrous 126W, but that must be less with only four nodes I think) they draw or how easy it's likely to be to make them silent. I'll probably be able to delete one or two 1gbit switches if I do all ethernet over 40gbe for the workstations, but those only pull about 7W each. The PC's are on an external watercooling loop so I could conceivably watercool the switch, though that's likely to beat least as much of hassle as noctua fans though it's considerably more likely I will be able to preserve the 1U profile that way.

Hopefully someone has a suggestion on the best of all worlds (compromise) approach to this?
 

i386

Well-Known Member
Mar 18, 2016
2,654
773
113
32
Germany
For 1-4 40GBE I would look at the icx switches in the megethread,
for 8+ 40GBE ports I would look at arista switches. The os of arista switches has a build in command to set the fans to 30% speed which makes them quieter than supermicro 3/4u servers.
That's an unmanaged infinband switch, it can't be used for ethernet networks.
 

Rand__

Well-Known Member
Mar 6, 2014
5,600
1,228
113
If you are fine with 12 ports the 6012's are nice, they run ~ 30-50W depending on used modules (DACs vs active), # of ports etc. Probably need fan modding in your case though, but I dont think any affordable 40G switch will run silent without, thats just not what they are made for.
If you need 30+ ports there is a short version of the 6036 available (o/c more power draw, ~50W+ iirc)
 

MichalPL

Member
Feb 10, 2019
45
5
8
Cisco 3064PQ you can mod for new fans and will be totally silent, but It has only 4 QSFP+ ports and 48 SFP+. if I remember correctly 65W
 

solon

Member
Apr 1, 2021
44
2
8
Thanks everyone, I'll inventory all those and see what's avaiable at a fair price on ebay. First quick glances look like they're not quite at the prices I'd like. I thought there was a thread on here somewhere where someone had modded a qsfp+ infiniband switch to 40gbe. I'll have so start sifting through everything again.

In total I only really need 4 ports, server and 3 pc's. Don't seen any lcx stuff on ebay, and the arista and cisco stuff is at a price level that I'd really need to think about.

I don't mind modding the fans to make it silent, but as noctua fans aren't cheap either, it's nice if it can be avoided.

Anyway thanks for the suggestions, I'll post back here if I can find anything affordable in Europe. All this working from home really has put a different load on my home network.
 

Rand__

Well-Known Member
Mar 6, 2014
5,600
1,228
113
You can import a EMC based 6012/18 from the US, will be ~€200.
Sometimes you can get a 6036 for the same price in EU, depends on how urgent it is:)
 

solon

Member
Apr 1, 2021
44
2
8
No urgency to be honest. This is almost completely surplus to requirements project to keep me occupied. It was initially caused by annoyance that it took 10 minutes to copy a game install from my PC to my GF's. That led to looking at 10gbe RJ45, which led to noticed 40gb prices, which led to infiniband.... which led to a side goal of an rdma drive on SAN for game installs. It just seems elegant to network at 40gbe now that I mostly have everything anyway.

I'll keep my eye out. To be honest a 6012 looks best for my needs, and seems most likely to be available. Not terribly enthusiastic about the prices I'm seeing at the moment, cheapest one was around 700,- euros. I may still do it, but I'll shop around unless I find myself particularly annoyed by something that would be alleviated by having 40gbe.