Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[eduncan911]

These two Amp devices start audio synced and get async by time. Is there any setting to avoid something like this? Or should I focus on the devices?

I'm new to L3 switches as well, knowing the concept for some time though.

However, and I could be completely wrong here, the only "tuning" you can do with a switch is with larger packets called Jumbo Frames (and both devices must support it). IIRC, that only helps for very large file transfers - I don't see how that can help streaming though.

There's also some QoS tuning that can happen across VLANs. But if you haven't configured QoS (and you know you have because it required precise settings of what to prioritize in packets), then that's not an issue.

Network really wouldn't have much of an affect when it comes to eventually getting out of sync'd. This sounds more like application timing running on the devices.

IMO, start a new thread about this as I don't think it's network related.

 

[fohdeesha]

yeah, the switching latency in that switch is ~400 nanoseconds. Things like QoS and buffers don't even come into play until you start pegging the port capacity to 100%, which I seriously doubt you're doing. most likely something with the applications

 

[eduncan911]

What Router is everyone pairing these monsters with?

I'm in-between routers at the moment, having used Mirotik, Unifi USG, EdgeRouter, and rolling my own Linux versions with VirtualSwitch... I'm going to build my own again using custom hardware and Xen for isolation (like QubesOS). But for right now, I need a stop gap until I have the time.

Was recently using AdvancedTomato but development seems to have stalled since 2017 with no more shibby releases. Looking for something to stay patched.

I no longer have the USG which would have been dirt easy to match the Unifi APs I have again with all the VLANs.

Was thinking of picking up a Mirotik RouterBoard but kind of don't want to spend a lot of time configuring it. And used USGs on ebay are too much $$$ IMO.

 

[sth]

Inbox zero new year resolution!? :-D

 

[Zervun]

What Router is everyone pairing these monsters with?

I'm in-between routers at the moment, having used Mirotik, Unifi USG, EdgeRouter, and rolling my own Linux versions with VirtualSwitch... I'm going to build my own again using custom hardware and Xen for isolation (like QubesOS). But for right now, I need a stop gap until I have the time.

Was recently using AdvancedTomato but development seems to have stalled since 2017 with no more shibby releases. Looking for something to stay patched.

I no longer have the USG which would have been dirt easy to match the Unifi APs I have again with all the VLANs.

Was thinking of picking up a Mirotik RouterBoard but kind of don't want to spend a lot of time configuring it. And used USGs on ebay are too much $$$ IMO.

I'm using Untangle - u50xw (although I don't use the wireless on it except guest network, have Ubi APs off of it)

Untangle home license is $50 a year, and it is a very well polished UTM. Interface is great. IPS/IDS isn't as great as some other UTMs but it works fine. Overall a very polished feature set. You don't have to pay for the home license it just adds some more features.

You can of course spin your own untangle instead of buying their box, it is just a Qotom. I've replaced the spinner with an SSD. Took me less than 30min as the restore function works fantastic.

 

[ronclark]

What Router is everyone pairing these monsters with?

I'm in-between routers at the moment, having used Mirotik, Unifi USG, EdgeRouter, and rolling my own Linux versions with VirtualSwitch... I'm going to build my own again using custom hardware and Xen for isolation (like QubesOS). But for right now, I need a stop gap until I have the time.

Was recently using AdvancedTomato but development seems to have stalled since 2017 with no more shibby releases. Looking for something to stay patched.

I no longer have the USG which would have been dirt easy to match the Unifi APs I have again with all the VLANs.

I am running Pfsence on a Dell R220 with a Intel quad nic.
it been about a year since I installed it. it's a big upgrade from my consumer Asus router.


Was thinking of picking up a Mirotik RouterBoard but kind of don't want to spend a lot of time configuring it. And used USGs on ebay are too much $$$ IMO.

I am running a Dell R220 with Pfsence.
it's been running great a huge upgrade from consumer router

 

[eduncan911]

I am running a Dell R220 with Pfsence.
it's been running great a huge upgrade from consumer router

Yeah I've ran pfSense on my UP Board Squared SoCs for a while and they are great. I just feel dirty running something with that much PHP in it.

I take it you have an add-in card for multiple 10 Gbps links? That was my plan for my next custom Linux build to have speed across the VLANs.

 

[ronclark]

Yeah I've ran pfSense on my UP Board Squared SoCs for a while and they are great. I just feel dirty running something with that much PHP in it.

I take it you have an add-in card for multiple 10 Gbps links? That was my plan for my next custom Linux build to have speed across the VLANs.

I have just Add on card and that's for my intel quad gigabit card. I just wanted some thing better than the built in Broadcom nic. if I move to another system and move the intel nic I dont have reconfigure all the ports.

 

[fohdeesha]

Yeah I've ran pfSense on my UP Board Squared SoCs for a while and they are great. I just feel dirty running something with that much PHP in it.

I take it you have an add-in card for multiple 10 Gbps links? That was my plan for my next custom Linux build to have speed across the VLANs.

just handle inter-vlan routing in hardware on the switch, no reason your firewall should be seeing 10gbE traffic

 

[CED6688]

Yes. For basic firewalling and NAT you can use just about anything.

I have 1gb symmetric fiber to the home and was getting near line speed with a cheap $70 EdgeRouter lite and have done the same with a Unifi Security Gateway (USG). I switched to a Mikrotik RB4011 because it provides NetFlow and IPSec at line-speed for cheap. I have a separate box running Suricata and Zeek (Bro).

If you are looking for in-line IPS over a few hundred Mb/s, you'll need something beefier, but otherwise, you just don't need much. Assuming you do all of your inter-VLAN routing on the ICX switches, the Traffic never hits one of these devices unless it is to/from the Internet...

 

[eduncan911]

just handle inter-vlan routing in hardware on the switch, no reason your firewall should be seeing 10gbE traffic

Yes. For basic firewalling and NAT you can use just about anything.
...
Assuming you do all of your inter-VLAN routing on the ICX switches, the Traffic never hits one of these devices unless it is to/from the Internet...

But what if you are opening/restricting specific TCP ports across the VLANs? E.g., I want to limit VLAN20 to only access VLAN10 IPs over port 443 and nothing else. Isn't that a Layer4/router-level firewall rule outside of the switch?

Or, are you saying routing all VLAN20 client traffic -> VLAN10 hosts (in switch), and restrict access on a per host basis for VLAN20 traffic? That way, no router is involved. Rather not do that with all of the docker containers and host machines I have...

Maybe I am mis-understanding the power of Layer 3 switches and specific ports?

Example use case: Wireless client connected to Guest network on VLAN20 (using Unifi AP and multiple VLANs tagged on a single switch port) trying to access intranet web server on VLAN10 (untagged host port VLAN10) - which I want to restrict to only port 443 access.

 

[fohdeesha]

you use ACLs https://forums.servethehome.com/ind...gbe-40gbe-switching.21107/page-36#post-205237

 

[JoshDi]

What Router is everyone pairing these monsters with?

I'm in-between routers at the moment, having used Mirotik, Unifi USG, EdgeRouter, and rolling my own Linux versions with VirtualSwitch... I'm going to build my own again using custom hardware and Xen for isolation (like QubesOS). But for right now, I need a stop gap until I have the time.

Was recently using AdvancedTomato but development seems to have stalled since 2017 with no more shibby releases. Looking for something to stay patched.

I no longer have the USG which would have been dirt easy to match the Unifi APs I have again with all the VLANs.

Was thinking of picking up a Mirotik RouterBoard but kind of don't want to spend a lot of time configuring it. And used USGs on ebay are too much $$$ IMO.

I use a pfSense box I built with an i7 and some i340-t4, i350-t2 and x550-t2 intel network cards.

 

[eduncan911]

you use ACLs https://forums.servethehome.com/ind...gbe-40gbe-switching.21107/page-36#post-205237

I haven't received my switch yet (7250-48P) so I can't explore the commands.

But in that link, it shows just IP permit/deny, no TCP Ports.

Are you saying TCP ports are another parameter?

 

[fohdeesha]

anything is a parameter. source port, destination port, packet protocol, etc

 

[CED6688]

I haven't received my switch yet (7250-48P) so I can't explore the commands.

But in that link, it shows just IP permit/deny, no TCP Ports.

Are you saying TCP ports are another parameter?

The best thing to look at for this will be the Security Configuration Guide. You can register and download for free on the Ruckus site. Look for "Extended ACL Lists", as those allow you to add L4 source/destination information. To say that the use of these ACLs is flexible is an understatement... and unlike a consumer router, applying many/complex ACLs has zero performance hit as it is all done in hardware.

Note that the commands for creating the lists change slightly between 08030->08092 for extended ACL lists, so be sure to grab the correct version of the manual.

The only rules I run on my actual firewall are SNAT/DNAT rules and a few filter rules to drop virtually everything coming in that are not via the VPN (I had OpenVPN running behind the firewall, but dropped it and just went with IKEv2+IPSec on the router itself w/ pub key auth) or SSH (pub key auth only).

 

[eduncan911]

And here I thought these were just plain ol' Layer 3 switches. Silly me!

 

[fohdeesha]

And here I thought these were just plain ol' Layer 3 switches. Silly me!

this is all quite standard for layer 3 switches (aside from maybe the mikrotik stuff)

 

[dashpuppy]

This might be a really big long shot, but does anyone have a spare set, or for sale pair of rack ears for the Brocade 6450 switch ?

 

[legen]

Quick question. I want to grab this ICX6450-24P : Brocade ICX6450-24P 24-Port Gig PoE Switch + 4 X SFP Ports (2 are SFP+ Ports) | eBay

The auction says "4 X SFP Ports (2 are SFP+ Ports)". Checking the awesome FAQ by fohdeesha https://fohdeesha.com/data/other/brocade/ICX6450 FAQ.pdf it says,

"A license (ICX6450-2X10G-LIC-POD) to upgrade the 1 GbE ports to 10 GbE speed is available and provides four 10 GbE ports per Brocade ICX 6450 Switch"

I just wanted to confirm that this will enable SFP+ and 10GbE speed on all fours ports for this specific switch i was looking at?

Thanks!