I want some QSFP+ switch, what should I look at?

tinfoil3d

QSFP28
May 11, 2020
338
121
43
Japan
Looking for some qsfp+ switch. The more qsfp+ the better. At least 6 is needed.
Someone said 7050QX-32S but that has atom with a bug, right? I'd like to avoid that. Chances of that failing are statistically higher.
Was looking at 7750 instead but they're both are quite dated right so no new fw?
Then the most important thing is what about licensing? icx and arista seem to be okay. What if I go mellanox? Or dell? Or cisco? Would that break my mind?
I would love the cheapest thing of course but it has to be reliable. And I'm still bad with screamers so the quieter it is the better. I don't need fanless because lol, they probably don't exist for such high speed devices. Arista seems to be good on that aspect too.
I'm open to suggestions, please help me navigate this crazy world of switches.
 

LodeRunner

Active Member
Apr 27, 2019
225
98
28
Looking for some qsfp+ switch. The more qsfp+ the better. At least 6 is needed.
Someone said 7050QX-32S but that has atom with a bug, right? I'd like to avoid that. Chances of that failing are statistically higher.
Was looking at 7750 instead but they're both are quite dated right so no new fw?
Then the most important thing is what about licensing? icx and arista seem to be okay. What if I go mellanox? Or dell? Or cisco? Would that break my mind?
I would love the cheapest thing of course but it has to be reliable. And I'm still bad with screamers so the quieter it is the better. I don't need fanless because lol, they probably don't exist for such high speed devices. Arista seems to be good on that aspect too.
I'm open to suggestions, please help me navigate this crazy world of switches.
My DCS-7050QX-32S reports that it is a "AMD GX-420CA SOC with Radeon(tm) HD Graphics" according to /proc/cpuinfo, so not subject to Intel's AVR54 bug.
 

Rand__

Well-Known Member
Mar 6, 2014
5,653
1,248
113
SX6012? Not really quiet out of the box o/c but cheap (if converted) and reliable...
 

Joshh

Member
Feb 28, 2017
41
6
8
40
7050-QX-32 is a great switch, the fact that you can lower the fan speed via command line is nice... makes it somewhat homelab friendly. I think the Atom bug is a bit hyped... its legit and I get why Hyperscalers would not want one in their environment but alot folks have them and have not had an issue. If you are interested, pm me. I have an extra one that is a reverse airflow model.
 

tinfoil3d

QSFP28
May 11, 2020
338
121
43
Japan
@LodeRunner thanks. There's a 700(800 w ship) offer on ebay, do you think it's a good deal for this?
@Rand__ is sx6012 a lot of work to convert to eth? Is it screaming? Breakout-friendly in eth version?
@Joshh sent. It's homelab at the moment simply because it's at home but majority of it is for business needs. And so downtime isn't tolerated much.
 

Rand__

Well-Known Member
Mar 6, 2014
5,653
1,248
113
The Ethernet conversion is included if you get an EMC switch and convert that to non EMC. That is a bit of work, couple of hours.
Helps if you know Linux, but not needed. Being able to read and follow instructions usually suffices;)

Or you can get one from MLX (Lenovo, HP) (more $$), no conversion needed but creative guessing of the necessary license key to enable Ethernet.
Both ways have been discussed at length in various threads and been done by many other ppl (some of which are happy to help;))

Noise - its not too bad, i have one in the basement, but wouldnt want to work next to it.
Breakout friendly? Absolutely:
1637872911091.png
 
  • Like
Reactions: tinfoil3d

fohdeesha

Kaini Industries
Nov 20, 2016
2,300
2,406
113
30
fohdeesha.com
the 7050QX-32S is an incredibly powerful and flexible switch for what you can get them for on ebay, upgradable RAM, SSD, can run docker containers and KVM VMs directly on the thing, arista's EOS is mostly just fedora and you can make them do pretty much anything you'd ever need to. the "S" variant will still run the latest EOS as well. It's an AMD SoC for the management plane so it's not affected by the Atom bug

on the same note I wouldn't say the Atom bug was overhyped either, it's definitely a problem and worth avoiding if you can, I and a lot of other STH users had to throw away Celestica 40gbE switches because of it. it's only going to happen at a higher rate as effected switches age more as well
 

tinfoil3d

QSFP28
May 11, 2020
338
121
43
Japan
@fohdeesha that's true. I love when linux is accessible and highly customizable(not fully in overlay/tmpfs).
And Layla recently mentioned hers 7060 has 10 SM qsfp28 on 30% fan speed and power consumption looks good too. Pretty amazing.
Do you know for sure these would work with any module I'd plug in? Be it mellanox AOC, be it cisco 40/100 sr-bidi, be it cisco 40 bidi? Or SM. Or "compatible" well-known brand modules? Because right now it's still at home but in future that would be in separate buildings(with one more similar unit), so need to minimize the amount of fibers, hence more expensive transceivers.
It doesn't stack right?

And yes, I mean, why would anyone deliberately step on a rake with atom bug, especially if it's for business. Even for home use, doesn't make any sense unless you're really low on budget. But again, lol, that may all just pop one day. And you'd still have to pay for a better choice.
 

klui

Active Member
Feb 3, 2019
431
181
43
Arista has transceiver locks and unlock codes are tied to an organization. The ones that are floating around work for older versions of EOS and not the latest ones. If you really intend to use the switches for business your bigger problem is lack of support contract if things go wrong.

For me dealing with the AVR54 bug for a home network isn't a big deal so long as the risk is understood. If it's for business it's actually a no brainer because your support contract affords you a replacement at no additional charge. I would never run any equipment for a business without a support contract.
 

tinfoil3d

QSFP28
May 11, 2020
338
121
43
Japan
I would never run any equipment for a business without a support contract.
It's okay, I evaluated the risks. My biggest problem would actually be simply either swapping the thing or waiting for another unit. Parts of network that need to be HA are HA anyway and will continue to run regardless. Maybe slower but still run. It's all still on SFP+ infra and some direct connections without switching.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,300
2,406
113
30
fohdeesha.com
Arista has transceiver locks and unlock codes are tied to an organization. The ones that are floating around work for older versions of EOS and not the latest ones. If you really intend to use the switches for business your bigger problem is lack of support contract if things go wrong.

For me dealing with the AVR54 bug for a home network isn't a big deal so long as the risk is understood. If it's for business it's actually a no brainer because your support contract affords you a replacement at no additional charge. I would never run any equipment for a business without a support contract.
have codes that definitely work in the latest EOS versions
 
  • Like
Reactions: klui and tinfoil3d

necr

Member
Dec 27, 2017
85
30
18
121
SX6012 (the most compact apart from being cheap), requires the most tinkering. Arista seems to be okay for your needs. High-end switches supporting SONiC are either expensive (SN2010/2100) or loud (Celestica).
 
  • Like
Reactions: tinfoil3d

tinfoil3d

QSFP28
May 11, 2020
338
121
43
Japan
Thank you. Will likely get arista. The only drawback I see with it is apparent lack of stacking(such a cool thing in brocade, especially without proprietary stacking cables).
 

klui

Active Member
Feb 3, 2019
431
181
43
have codes that definitely work in the latest EOS versions
That's encouraging. I've read about folks who tried the service command and found it was unrecognized only to discover they were probably older models of the switches that use the enable3px empty file instead of the command. I haven't had a need to use the command on my 7060cx-32s.

 

fohdeesha

Kaini Industries
Nov 20, 2016
2,300
2,406
113
30
fohdeesha.com
SX6012 (the most compact apart from being cheap), requires the most tinkering. Arista seems to be okay for your needs. High-end switches supporting SONiC are either expensive (SN2010/2100) or loud (Celestica).
the arista has full sonic support https://www.servethehome.com/get-started-with-40gbe-sdn-with-microsoft-azure-sonic-for-under-1k/
 

tinfoil3d

QSFP28
May 11, 2020
338
121
43
Japan
Thanks @i386 . I'm just pretty new to all this enterprise switching stuff lol. I only learned about what's sfp i think this year. I since purchased multiple 10gig switches from different vendors and got comfortable in their very similar CLI all just with TABs so still a lot to learn.
 

necr

Member
Dec 27, 2017
85
30
18
121
the arista has full sonic support https://www.servethehome.com/get-started-with-40gbe-sdn-with-microsoft-azure-sonic-for-under-1k/
you're right, 7050QX-32S is still in the HCL. Just haven't seen any posts about running SONiC on these recently. High-end I meant something current/interesting that can do RIOT and better telemetry (Trident 3/Spectrum/Tomahawk2)