Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[kpfleming]

Yes. Any VLAN without a VE will be a layer 2 VLAN.

 

[kpfleming]

Hello everyone, I'm building a new house and I try to decide for ICX6610-48P or ICX7450-48P or two 24 port switches in stack (PoE and not-PoE)

I know that my question is already present al lot of time, but I didn't find the right answer.

My needs are:

- POE+ ports for APs, VOIP Adapters, IP Cameras, maybe POE Lighting system with sensors, Video Entryphone and any other device that I can connect (also with PoE splitter) by wire instead of WIFI
- Three 10GB and 12 1Gb connections to my little cluster (2 esxi box, 1 nas) and my workstation (with 10base-t transceiver)
- 16 1GB devices (PCs, RPIs, AV Receiver, Video Projector, TV, SAT->IP...)
- Dual PSU protected by UPS to prevent any possible downtime for critical devices (Alarm, Camera IP, Door Bell, Video Entryphone and so on)
- when I will go to live in, a big media center (now I'm using Emby on my Workstation, but it is full of HDDs) like this one: 4U 24 Bay SAS3 Vmware Storage Server X10QBi Includes CPU/Memory
- power drain is not a a problem
- noise is not a problem (I will dedicate a room in basement)


My concern is about the pro and cons between the two units:
ICX 6xxx is EOL and ICX7xxx is supported, so it means that the first one had the last firmware, while the latter will have new firmware (now 9.0).
But is the new firmware a worth update? Which new features does it have?
ICX7xxx seems to have less port 10GB or 40 GB ports
ICX7xxx have also PoH support (90W), but I didn't find a device that use it. Also the last Wifi 6 Ap uses the PoE+ standard. Anyone could give me a list of these devices?

After all, I ask everyone to convince me which one is the best or alternative solutions.

I'm not in hurry, because the house will be ready and the end on the year

Thank you very much

Given the size of this thread, it might best if you started a separate thread for this specific discussion... otherwise it will get lost in the mix with everything else here.

 

[tubs-ffm]

Is it possible to just have VEs for MGMT and HOME, and have those routed on the switch with no ACLs, but keep forcing my other VLANs to go out to the firewall?

Yes. Any VLAN without a VE will be a layer 2 VLAN.

Add VE to the VLAN you want to have L3 routing in between.
And without VE it is L2 switching, what means no routing.

I did similar here but was looking for additional control by filters.

https://forums.servethehome.com/index.php?threads/ruckus-brocade-acl-l3-default-behavior.31723/post-294921

 

[adman_c]

Yes. Any VLAN without a VE will be a layer 2 VLAN.

Add VE to the VLAN you want to have L3 routing in between.
And without VE it is L2 switching, what means no routing.

I did similar here but was looking for additional control by filters.

https://forums.servethehome.com/index.php?threads/ruckus-brocade-acl-l3-default-behavior.31723/post-294921

Great. I already have my firewall rules setup the way I want for my isolated VLANs. This way I don't need to mess with ACLs--MGMT and HOME can talk to each other freely as needed, and at line speed.

 

[clcorbin]

This may be a function of my extremely poor search skills, but I only found 3 posts in this thread talking about 2.5 Gb speeds. And none of them actually answer my question. So...

Can the x/3/x ports on the ICX 6610 support NBase-T copper (2.5Gb in my case) with the right SFP+ transceiver? I'm getting a new cable modem that supports that link speed (old modem appears to be overheating and downshifting to 100mbps) and while I don't need >gigabit speeds yet, it is only a matter of time before I do.

Thanks!

 

[Balteck]

This may be a function of my extremely poor search skills, but I only found 3 posts in this thread talking about 2.5 Gb speeds. And none of them actually answer my question. So...

Can the x/3/x ports on the ICX 6610 support NBase-T copper (2.5Gb in my case) with the right SFP+ transceiver? I'm getting a new cable modem that supports that link speed (old modem appears to be overheating and downshifting to 100mbps) and while I don't need >gigabit speeds yet, it is only a matter of time before I do.

Thanks!

This thread answers your question: https://forums.servethehome.com/ind...t-marvell-88x3300-v-s-aquantia-aqs-107.30004/

 

[mintchipmadness]

Hello All,
I am trying to setup separate wlans on my access point (ruckus r710) and wanted to see if my switch (icx 7250) setup is the problem. I currently have 3 wlans setup on the access point (AP). 1 wlan is untagged (main) and the other two are tagged (10 and 15 respectively). In the end I would like the tagged wlans to go on separate networks (192.168.5.0 and 192.168.10.0) with the untagged going to my main lan (192.168.1.0). I setup the same vlans on the switch and tagged both vlans to the port of the AP. Just in case it is the problem, I created a lag from the access point to switch (to learn how) so the port I tagged on the new vlans is lag 1. Overall, when I connect to the tagged wlans I cannot get past the AP. Is my switch setup the issue? All I did was setup the vlans and tag the ports. Should I have setup virtual interfaces for each vlan? I tried that too and I get the same result. I am guessing the issue is the router (OPNsense) but I wanted to make sure it wasn't the switch first. Thank you for your help.

 

[tubs-ffm]

This way I don't need to mess with ACLs--MGMT and HOME can talk to each other freely as needed, and at line speed.

It depends what you want to achieve. In my case I wanted to use the 10 GBit ports of my switch to route between LAN and DMZ. If I go via my firewall, I would create 1 GBit bottleneck. But ACLs where required in my case. Otherwise having a split in the two networks LAN and DMZ would be meaningless if unlimited routing will be possible.

 

[kpfleming]

Hello All,
I am trying to setup separate wlans on my access point (ruckus r710) and wanted to see if my switch (icx 7250) setup is the problem. I currently have 3 wlans setup on the access point (AP). 1 wlan is untagged (main) and the other two are tagged (10 and 15 respectively). In the end I would like the tagged wlans to go on separate networks (192.168.5.0 and 192.168.10.0) with the untagged going to my main lan (192.168.1.0). I setup the same vlans on the switch and tagged both vlans to the port of the AP. Just in case it is the problem, I created a lag from the access point to switch (to learn how) so the port I tagged on the new vlans is lag 1. Overall, when I connect to the tagged wlans I cannot get past the AP. Is my switch setup the issue? All I did was setup the vlans and tag the ports. Should I have setup virtual interfaces for each vlan? I tried that too and I get the same result. I am guessing the issue is the router (OPNsense) but I wanted to make sure it wasn't the switch first. Thank you for your help.

It sounds like you have the switch setup properly for the link to the AP. Now you need to do a similar configuration for the link between the switch and the router: the port(s) will need to have tagged VLANs 10 and 15 on them, and the router will need to have its own virtual interfaces so that it can accept traffic, provide addresses via DHCP (if you use it), etc.

Also, while you're still working on it, please consider using VLAN tags and subnet numbers that match, if you can. Otherwise you'll have to remember that VLAN tag 10 is subnet 5, and VLAN tag 15 is subnet 10. The network devices won't care, but future you could easily be confused by having '10' mean something different in those two cases.

 

[tubs-ffm]

Is my switch setup the issue? All I did was setup the vlans and tag the ports. Should I have setup virtual interfaces for each vlan? I tried that too and I get the same result. I am guessing the issue is the router (OPNsense) but I wanted to make sure it wasn't the switch first.

Hard to follow the verbal description. Any sketch from your network would help.

I suggest to first skip LAG and get VLAN running. At the moment it is unclear if the issue is in the setup of VLAN or LAG or router.

Basically I am doing the same as you with R710, ICX 7150 and OPNsense. Separate WiFi SSID with separate VLANs: "Home" (1), "Guest" (20) and IoT" (30) . On ICX VLAN 1 untaged and VLAN 20 and 30 tagged. L2 set-up on ICX, means no VE assigned only to VLAN 1. Network defined on OPNsense.

vlan 1 name DEFAULT-VLAN by port
!
vlan 10 name DMZ by port
 tagged ethe 1/2/1
 untagged ethe 1/1/3 ethe 1/1/5 ethe 1/3/3 to 1/3/4
!
vlan 20 name IoT by port
 tagged ethe 1/1/1 ethe 1/2/1
 untagged ethe 1/1/11
!
vlan 30 name Guest by port
 tagged ethe 1/1/1 ethe 1/2/1 ethe 1/3/3
 untagged ethe 1/1/7 ethe 1/1/9
!
!
!
!
ip route 0.0.0.0/0 192.168.2.1
!
!
!
!
interface ethernet 1/1/1
 port-name AP
!
 interface ethernet 1/2/1
 port-name OPNsense
!
!
!
!
interface ve 1
 ip address 192.168.2.2 255.255.255.0
 ipv6 address fd00:0:0:2::2/64

 

[mintchipmadness]

It sounds like you have the switch setup properly for the link to the AP. Now you need to do a similar configuration for the link between the switch and the router: the port(s) will need to have tagged VLANs 10 and 15 on them, and the router will need to have its own virtual interfaces so that it can accept traffic, provide addresses via DHCP (if you use it), etc.

Also, while you're still working on it, please consider using VLAN tags and subnet numbers that match, if you can. Otherwise you'll have to remember that VLAN tag 10 is subnet 5, and VLAN tag 15 is subnet 10. The network devices won't care, but future you could easily be confused by having '10' mean something different in those two cases.

Thank you for your reply and your advice on the subnets. I definitely agree the subnets should match the vlans and that is how I set it up. The 192.168.5.0 should have been typed 192.168.15.0. Too fast typing on my part.

 

[mintchipmadness]

Hard to follow the verbal description. Any sketch from your network would help.

I suggest to first skip LAG and get VLAN running. At the moment it is unclear if the issue is in the setup of VLAN or LAG or router.

Basically I am doing the same as you with R710, ICX 7150 and OPNsense. Separate WiFi SSID with separate VLANs: "Home" (1), "Guest" (20) and IoT" (30) . On ICX VLAN 1 untaged and VLAN 20 and 30 tagged. L2 set-up on ICX, means no VE assigned only to VLAN 1. Network defined on OPNsense.

vlan 1 name DEFAULT-VLAN by port
!
vlan 10 name DMZ by port
tagged ethe 1/2/1
untagged ethe 1/1/3 ethe 1/1/5 ethe 1/3/3 to 1/3/4
!
vlan 20 name IoT by port
tagged ethe 1/1/1 ethe 1/2/1
untagged ethe 1/1/11
!
vlan 30 name Guest by port
tagged ethe 1/1/1 ethe 1/2/1 ethe 1/3/3
untagged ethe 1/1/7 ethe 1/1/9
!
!
!
!
ip route 0.0.0.0/0 192.168.2.1
!
!
!
!
interface ethernet 1/1/1
port-name AP
!
interface ethernet 1/2/1
port-name OPNsense
!
!
!
!
interface ve 1
ip address 192.168.2.2 255.255.255.0
ipv6 address fd00:0:0:2::2/64

Thank you for your help. I will try to remove the lag and see if that works because everything else is the same as your setup. For the network sketch my setup is pretty straight forward. It goes AP (r710)--> Switch (icx 7250) -->Router (OPNsense). AP is plugged into the switch through 1/1/37 and 1/1/39 (lag 1) and the router is plugged into the switch through 1/2/1. All ports are untagged on the default vlan with interface ve 1 and ip address 192.168.1.2. I did notice one difference between your config and mine. It is "ip route 0.0.0.0/0 192.168.2.1". Would you be able to provide some detail on what that does? I am wondering if that is what I am missing. Thank you.

 

[tubs-ffm]

I did notice one difference between your config and mine. It is "ip route 0.0.0.0/0 192.168.2.1". Would you be able to provide some detail on what that does? I am wondering if that is what I am missing.

192.168.2.1 is my OPNsense on LAN network. ICX is 192.168.2.2. This line is to tell the ICX the route to find the way to the firewall/router. Not needed in your network for other devices to find the way. You either have setup the default route manually on each device or each device get the default route by DHCP of OPNsense. The ICX need this line it that services running on ICX, like NTP, can find the way to the router.

 

[adman_c]

It depends what you want to achieve. In my case I wanted to use the 10 GBit ports of my switch to route between LAN and DMZ. If I go via my firewall, I would create 1 GBit bottleneck. But ACLs where required in my case. Otherwise having a split in the two networks LAN and DMZ would be meaningless if unlimited routing will be possible.

Yeah, I'm content with my firewall being the bottleneck for those instances when I need to route between a trusted network and a non-trusted one. Those instances are relatively infrequent. Plus I'm building a tiny-mini firewall with 10gbe and a beefier CPU to replace my celeron J3160, so that bottleneck should be lessened as well. Mostly I was curious if things would get confused if the switch knows how to route to *some* local subnets but not others.

 

[mintchipmadness]

192.168.2.1 is my OPNsense on LAN network. ICX is 192.168.2.2. This line is to tell the ICX the route to find the way to the firewall/router. Not needed in your network for other devices to find the way. You either have setup the default route manually on each device or each device get the default route by DHCP of OPNsense. The ICX need this line it that services running on ICX, like NTP, can find the way to the router.

Thank you for that explanation. I will continue to troubleshoot this issue.

 

[adman_c]

Not sure if this is much of a good deal, but I came across this listing on ebay for anyone looking for a ICX7450-48p for $400 OBO:

Brocade (ICX725048P2X10G) 48 Port Ethernet Switch for sale online | eBay

Find many great new & used options and get the best deals for Brocade (ICX725048P2X10G) 48 Port Ethernet Switch at the best online prices at eBay! Free shipping for many products!

www.ebay.com

Honestly though the 7450 seems like such an odd switch to me. It's so much less capable than the 6610, but just as loud/power hungry. I mean, it doesn't even have as many 10gb SFP+ as the 7250! I guess if you really, really need a few 802.3bt ports for something?

 

[fohdeesha]

Not sure if this is much of a good deal, but I came across this listing on ebay for anyone looking for a ICX7450-48p for $400 OBO:

Brocade (ICX725048P2X10G) 48 Port Ethernet Switch for sale online | eBay

Find many great new & used options and get the best deals for Brocade (ICX725048P2X10G) 48 Port Ethernet Switch at the best online prices at eBay! Free shipping for many products!

www.ebay.com

Honestly though the 7450 seems like such an odd switch to me. It's so much less capable than the 6610, but just as loud/power hungry. I mean, it doesn't even have as many 10gb SFP+ as the 7250! I guess if you really, really need a few 802.3bt ports for something?

yeah, there's a reason it's not in my main post, it has so many little hidden limitations, and fully loaded it can't even match the bandwidth on the 6610's rear ports alone, while running the same fans and PSUs. not to mention the 10gbe modules are at insane prices these days (like, $400 each or more), so the only way to really get a usable config is to buy one that already has a module like the one you linked. I've specd them for a couple clients who had very specific needs that the 8030 codetrain on the 6610 couldn't fill in colo like mss clamping in hardware to accommodate gre tunnels, etc, but for home use meh

 

[NablaSquaredG]

7150-C12 have almost doubled in price... Have you Americans finally realised that power is not free?

Does anyone have an ICX7150-12 for sale for a reasonable price?

 

[Balteck]

yeah, there's a reason it's not in my main post, it has so many little hidden limitations, and fully loaded it can't even match the bandwidth on the 6610's rear ports alone, while running the same fans and PSUs. not to mention the 10gbe modules are at insane prices these days (like, $400 each or more), so the only way to really get a usable config is to buy one that already has a module like the one you linked. I've specd them for a couple clients who had very specific needs that the 8030 codetrain on the 6610 couldn't fill in colo like mss clamping in hardware to accommodate gre tunnels, etc, but for home use meh

So 8.0.40+ or 9.0.10 codetrain don't offer much more that is worth upgrading ?

I think about IPv6, configuration archive and compare (like Junos) and much more I don't know...

 

[Rttg]

7150-C12 have almost doubled in price...

Yeah, it’s absolutely nuts. I managed to pick one up 6mo ago for $200 - thought about buying another for stacking on the other side of house, but now the cheapest I can find is ~$375.

If you can get by without 10Gbe (and with the older 08.0.30 code train), an ICX6450-C12-PD might be the ticket