Currently it's 450$
.What would be the use case to install SONIC instead of OpenWRT on these, when used as a HomeLab Network Switch ?
Celestica Seastone DX010 32Port 100G QSFP28 $250
- Thread starter Karpo
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
.What would be the use case to install SONIC instead of OpenWRT on these, when used as a HomeLab Network Switch ?
To get data from port A to port B (switching)
But seriously it's about having more up to date software compared to running a vendor os/firmware that is using outdated software (for example openssh server with known exploits), be it Sonic, Opewrt or other "nos" that might have slightly different goals/users. (The functionality/featureset is dictated by the used hardware and the asics do not change, so software won't get you more features)
Sonic is much worse to administrate than other Network Operating Systems. Manually configuring a yaml file is prone to errors. Most commands do not work and most people with Sonic just run it as L2 switch.
There’s a reason why these switches are often comparatively cheap:
The lack of a good NOS.
SONiC is not a good NOS.
OpenWRT does not run on any big enterprise network switch (okay, maybe Mellanox, because these have mainline Linux kernel support), and is not a good NOS.
Oh and of course:
The ASIC is a Broadcom Tomahawk 1, which has lots of limitations.
And the switches have CPUs that are potentially affected by the AVR54 Atom C2000 bug.
The lack of a good NOS.
SONiC is not a good NOS.
OpenWRT does not run on any big enterprise network switch (okay, maybe Mellanox, because these have mainline Linux kernel support), and is not a good NOS.
Oh and of course:
The ASIC is a Broadcom Tomahawk 1, which has lots of limitations.
And the switches have CPUs that are potentially affected by the AVR54 Atom C2000 bug.
Yeah, if you can wait, you will probably see good 100G switches below 500USD the next 12-18 months. In the meantime, just get either Mellanox SX6036 or Arista 7050QX-32(S). Both are awesome. Mellanox SX6036 also support 56G. Even if SSH is outdated, you do not run them on the internet. Preferably you have a seperate physical network to access these.
Depends what you define as „good“
Juniper QFX5200 (Tomahawk) are already around $500
Trident 3 switches below $500? Some models already are, but only Whitebox. Proper ones like Arista 7050X3 or Juniper QFX5120 below $500? We‘ll see.
Mellanox SN2000 is not quite there yet.
Not familiar with the Juniper QFX5200, if you guys says its good, then it is probably good. My requirements is mostly related to RoCE.
Same chip as 7060CX, DX010, Z9100, etc…
But good NOS (JunOS). Not quite as good as Arista, but definitely A-Tier.
But good NOS (JunOS). Not quite as good as Arista, but definitely A-Tier.
Currently it's 450$
What would be the use case to install SONIC instead of OpenWRT on these, when used as a HomeLab Network Switch ?
yo dawg lovingly if you think openwrt is even a remotely viable OS to put on enterprise ASIC based 100g switches then you probably shouldn't be shopping for them
Yeah, that makes it a very interesting switch for that price. Power consumption is still high for homelab though.
Thank you for the "useful" Tip.
From where I stand, OpenWRT > any proprietary "OS" like Zyzel or HP(E) etc on the SMB 1gbps Series (e.g. Zyxel GS1900). At least you don't get Paperweights with extremely outdated and vulnerable Firmware.
Since I never used SONIC, I believe it's a legitimate Question, which still isn't answered
.So, what do you get, in exchange for no GUI and CLI-only ? I would assume the SOC Support / ASIC Network Switch (plus possibly the Management CPU/SOC, if separate, e.g. X86 on Mellanox SN2700 I believe) is baked into the Linux Kernel (plus some Firmware), so Device Support should be similar, since both are based on Linux.
Granted I have a liking to Debian, so SONIC could be interesting as well
.Exactly are the Pros vs Cons of SONIC vs something like OpenWRT ?
That's also what's turning me Off. 100W+ Idle with not even a Port Used. Yikes
.
SONIC is maybe Debian under the hood. But you will quickly see how britle and fragile it is. As L2 switch it is ok, L3 can be hit and miss. I seems that OpenWRT works on Spectrum switches [OpenWrt Wiki] Mellanox Spectrum Switches but, you will lose important fucntionality. For example enabling ROCE is as easy as just writing: roce enable onGranted I have a liking to Debian, so SONIC could be interesting as well
Exactly are the Pros vs Cons of SONIC vs something like OpenWRT ?
I'm mainly interested in L2 Functionality (and Offloading) anyways.
I might be highly conservative but for me a Switch is Switching (L2), not Routing (L3). A Router is Routing (L3), not really intended to do Switching at Line Speed (L2).
All of these L2+ or L3 Light or L3 or whatever Switches (I assume mainly for Routing across VLANs) feels a bit "weird" to me, especially if you would like to have a Firewall in between your Subnets / VLANs (which you SHOULD definitively have).
I think your missing a big use case for these kinds of switches: HCI
I have a similar model with 6 machines hooked up to it, but each machine has 1-4 k8s clusters on it and each one talks BGP to the switch. I have 10.0.0.0/16 divided into /24s routed through it. In my case I'm not doing any acl stuff, but simple segmentation would be ready to do. Dumping that out of the switch and into a "firewall" would be silly.
I have zero Experience with that. I even had to look it up because the Acronym wasn't telling me anything
.Not sure why you have them talking BGP to the Switch. Couldn't you just do "normal" Routing with possibly Static Routes ?
I looked A BIT into BGP when I was considering getting some extra Public IP Addresses for my Homelab, but it didn't look very interesting/easy to setup (although in that case I needed a PUBLIC "Node" to make it work, like e.g. a VPS, Problem of course that you do NOT have in your Private IP Address Space). Not sure what would be the Advantages in such Case though ...
Hyper converged infrastructure.I have zero Experience with that. I even had to look it up because the Acronym wasn't telling me anything
Not sure why you have them talking BGP to the Switch. Couldn't you just do "normal" Routing with possibly Static Routes ?
And I have no interest in managing 256 static routes, especially since they can change dynamically. BGP + rest apis, I haven't logged into that switch in years.
You just aren't the target market for those features it seems.
The only limitations of Tomahawk 1 I'm aware are it can't do L3 EVPN/VXLAN. Those in the know also say 25G port-to-port latency aren't as good as Tridents. Do you know of others?
Atom limitations does not apply to all switches that use the Tomahawk 1. For example, Arista 7060CXs use AMD GX-424CC, Juniper QFX5200s use Xeon E312xx. Celestica DX010, definitely buyer beware but safe if manufactured after 2018.
It can, but only with recirculation. That’s just one of the limitations. Check the Arista Feature Matrix for 7050X(3) vs 7060X to get an impression
Tomahawk 1 has issues with FEC on breakout ports (no RS-FEC on 25G ports)
Sadly, most Tomahawk 1 switches have the Atom. Other examples are Dell Z9100 or Efge Core AS7712-32X