Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[Lone Wolf]

where do I donate to the @fohdeesha beverage fund?

The guide is a fantastic resource! There's a Donate link at the top of the page: Brocade Overview - Fohdeesha Docs

 

[thebwack]

The guide is a fantastic resource! There's a Donate link at the top of the page: Brocade Overview - Fohdeesha Docs

Done. Now about to try my hand at some paper clip MPO trunks.

thanks again!

I’ll post some pics of our setup when we’re done. Setting up some 40G uplinks for our Truenas box to serve video and rendering editors.

 

[fohdeesha]

Done. Now about to try my hand at some paper clip MPO trunks.

thanks again!

I’ll post some pics of our setup when we’re done. Setting up some 40G uplinks for our Truenas box to serve video and rendering editors.

Thank you!

 

[adman_c]

Done. Now about to try my hand at some paper clip MPO trunks.

thanks again!

I’ll post some pics of our setup when we’re done. Setting up some 40G uplinks for our Truenas box to serve video and rendering editors.

Sheeit. All this time and I've just been freeloading! Donated!

Anyone think we'll start to see more 7250 models showing up on ebay now that they've been emergency suddenly EOL'd discontinued? Or do we think that folks will hang on to them since there's no current equivalent for a 24/48 port switch with > 4 sfp+ slots without going all the way up to the 7450. (Edited that for more accuracy).

Oh, and sorry to bump my own question, but does anyone have any ideas why I can't seem to get my 7250 to route properly?

OK. I've been banging my head against this for a bunch of hours and I would love it if someone could just point out where I'm being dumb. I cannot for the life of me get my 7250 to route between VLANs. I'm running pfsense/opnsense (virtualized and switching between back and forth between the two while I get my opnsense install fully operational--for the purposes of this question, it doesn't matter which firewall I'm running). I have a bunch of VLANs--more than I need, but whatever. 3 of the VLANs are trusted, and I want to be able to route between them via the switch rather than going out to the firewall. The rest of the VLANs I want to go ahead and use the firewall to the extent there needs to be routing between them (rare), because I'm substantially more comfortable with filter rules than ACLs. At the bottom of this message is my current running config, and here is the output of 'sh ip route':

SSH@coreswitch(config)#sh ip route
Total number of IP routes: 6
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric
BGP  Codes - i:iBGP e:eBGP
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2
STATIC Codes - v:Inter-VRF
        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          172.16.2.1      ve 2162       1/1           S    10h49m
2       10.10.10.0/24      DIRECT          ve 1010       0/0           D    10h49m
3       172.16.1.0/24      DIRECT          ve 2161       0/0           D    10h49m
4       172.16.2.0/24      DIRECT          ve 2162       0/0           D    10h49m
5       192.168.0.0/24     DIRECT          ve 2          0/0           D    10h49m
6       192.168.10.0/24    DIRECT          ve 10         0/0           D    10h49m

The three VLANs between which I want to route are 2, 10, and 1010. 2161 and 2162 are transit VLANs for WAN and LAN, respectively. When I set the gateway on any of the trusted VLANs for the firewall (X.X.X.1), everything works as expected. The firewall routes between VLANs according to my rules and I can get out to the internet. On the other hand, when I set the gateway on any of the trusted VLANs for the switch (X.X.X.254), I cannot reach one subnet from another. SSH/HTTPS are both inaccessible between local subnets. However, going out to the internet works, and for some reason I can ping between local subnets. This behavior is the same whether I'm running pfsense and opnsense, and even if I yank the LAN transit cable between the switch and the firewall.

At this point I'm ready to give up and just let the firewall handle all the routing, even thought it's not quite up to the task of linespeed routing. As an aside, how much CPU do you need to max iperf on 10gbe? Brief testing I can get ~7gbit with my i3-8100t.

Anyway, any suggestions would be awesome.

Thanks!

SSH@coreswitch>sh run
Current configuration:
!
ver 08.0.95fT213
!
stack unit 1
  module 1 icx7250-24p-poe-port-management-module
  module 2 icx7250-sfp-plus-8port-80g-module
!
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
spanning-tree
!
vlan 2 name infra by port
tagged ethe 1/2/2 to 1/2/4
untagged ethe 1/1/2 to 1/1/3 ethe 1/1/5 to 1/1/7 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24
router-interface ve 2
spanning-tree
!
vlan 10 name home by port
tagged ethe 1/1/2 ethe 1/1/7 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24 ethe 1/2/2 to 1/2/4
untagged ethe 1/1/4 ethe 1/1/8 to 1/1/9 ethe 1/1/11 ethe 1/1/14 ethe 1/1/16 ethe 1/2/5
router-interface ve 10
spanning-tree
!
vlan 11 name voip by port
tagged ethe 1/1/2 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24 ethe 1/2/2
untagged ethe 1/1/12 ethe 1/1/18
spanning-tree
!
vlan 12 name guest by port
tagged ethe 1/1/2 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24 ethe 1/2/2
spanning-tree
!
vlan 20 name kids by port
tagged ethe 1/1/2 ethe 1/1/5 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24 ethe 1/2/2
spanning-tree
!
vlan 30 name IOT by port
tagged ethe 1/1/2 ethe 1/1/7 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24 ethe 1/2/2 to 1/2/4
untagged ethe 1/1/13 ethe 1/1/17 ethe 1/1/19 ethe 1/1/22 to 1/1/23 ethe 1/2/7
spanning-tree
!
!                                                             
vlan 999 by port
tagged ethe 1/1/24 ethe 1/2/2
!
vlan 1010 name data by port
tagged ethe 1/1/2 ethe 1/1/15 ethe 1/1/20 to 1/1/21 ethe 1/1/24 ethe 1/2/2
untagged ethe 1/2/3 to 1/2/4
router-interface ve 1010
spanning-tree
!
!
!
!
vlan 2161 name wansit_176_16_1 by port
untagged ethe 1/2/1
router-interface ve 2161
!
vlan 2162 name lansit_176_16_2 by port
untagged ethe 1/2/2
router-interface ve 2162
!
vlan 2222 name wan_vlan by port
tagged ethe 1/2/1
untagged ethe 1/1/1                                          
spanning-tree
!
!
!
vlan 3333 name 5g_wan_vlan by port
tagged ethe 1/1/24 ethe 1/2/1
untagged ethe 1/1/10
spanning-tree
!
!
!
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
hostname coreswitch
ip dhcp-client disable
ip dns server-address 192.168.0.1
ip route 0.0.0.0/0 172.16.2.1
!
no telnet server
username super password .....
!
!
!
!
clock summer-time
clock timezone gmt GMT-06
!
!
ntp
disable serve
server 192.168.0.1
server 10.10.10.1
!
!
no web-management http
web-management https
!
manager disable
!                                                             
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface management 1
ip address 10.10.2.254 255.255.255.0
!
interface ethernet 1/1/1
port-name cablemodem
!
interface ethernet 1/1/2
port-name firemox
!
interface ethernet 1/1/3
port-name prox-enp35
!                                                             
interface ethernet 1/1/4
port-name printer
!
interface ethernet 1/1/5
port-name minimox-eno1
!
interface ethernet 1/1/6
port-name piman
!
interface ethernet 1/1/7
port-name micromox1
!
interface ethernet 1/1/8
port-name IPMI1
!
interface ethernet 1/1/9
port-name note-nook
!
interface ethernet 1/1/12
port-name obi200
!
interface ethernet 1/1/15
port-name kitchen-no-poe                                     
!
interface ethernet 1/1/17
port-name master-bed
!
interface ethernet 1/1/18
port-name security
!
interface ethernet 1/1/19
port-name garage
!
interface ethernet 1/1/20
port-name foyer
!
interface ethernet 1/1/21
port-name kitchen
!
interface ethernet 1/1/22
port-name garage-south-2
!
interface ethernet 1/1/23
port-name 2nd-bed
!
interface ethernet 1/1/24                                     
port-name 4th-floor
!
interface ethernet 1/2/1
port-name WANuplink
!
interface ethernet 1/2/2
port-name LANuplink
!
interface ethernet 1/2/3
port-name mmx-10g
!
interface ethernet 1/2/4
port-name prox-10g
!
interface ethernet 1/2/5
port-name m1mini
!
interface ve 2
ip address 192.168.0.254 255.255.255.0
!
interface ve 10
ip address 192.168.10.254 255.255.255.0
!                                                             
interface ve 1010
ip address 10.10.10.254 255.255.255.0
!
interface ve 2161
ip address 172.16.1.254 255.255.255.0
!
interface ve 2162
ip address 172.16.2.254 255.255.255.0
!
!
!
!
!
!
!
!
!
!
ip ssh  password-authentication no
ip ssh  idle-time 0
ip ssh  interactive-authentication no
!
!                                                             
!
!
!
end

 

[zunder1990]

Anyone think we'll start to see more 7250 models showing up on ebay now that they've been emergency EOL'd? Or do we think that folks will hang

Do you have a source for that, we use those are work alot.

 

[adman_c]

Do you have a source for that, we use those are work alot.

ICX7250 | RUCKUS ICX 7250 Switches

Delivers gigabit Ethernet switch functionality and Layer 3 services without compromising performance and reliability

www.commscope.com

 

[LodeRunner]

I think that's just End of Sale, not End of Life/End of Support.

 

[adman_c]

I think that's just End of Sale, not End of Life/End of Support.

Oh shit you're right. I even read that when I saw that notice the first time. They're not EOL or EOS for a couple years at least. Sorry for the alarm! I edited my post. And also tracked down the doc with the dates.

 

[danb35]

And now a 6610 is here, updated, licensed, etc. with no problems. Outstanding. But damn, that's loud when it first starts up. Not too bad once the fans ramp down, but I don't think I want to be anywhere close if they have to spend any time at Speed 2.

 

[rootwyrm]

I think that's just End of Sale, not End of Life/End of Support.

Nope. It is an emergency EOS and permanent EOSL.

https://support.ruckuswireless.com/...ce=[Documentation]&f:mad:commonproducts=[EOL]

Last Order: prior to Feb 7, 2022
Last Delivery: prior to July 1, 2022
Prior EOSD: no plans to terminate software development
New EOSD: terminating 1 year after final shipment, no later than July 1, 2023 but will be sooner if supply is exhausted before July 1
Prior EOSL: 2030ish
Last Contract: no later than July 1, 2022 but will be sooner if supply is exhausted before then
Absolute Contract Termination: no later than July 1, 2027 (5yr)

Additionally, all support contracts and licenses have been permanently discontinued effective no later than July 1, 2022. I've also been told (don't have solid confirmation) that Ruckus stopped permitting any same-day hardware contracts either new or renewal in February. 4 hour software only, no hardware secure uplifts, and no 4 hour parts.

edit: to clarify, EOSD means that no 7150 or 7250 will receive any further software updates, including security fixes, after July 1, 2023. So the market may shortly be flooded as this has the immediate effect of requiring all GSA, FedRamp, and CMS providers to immediately replace every 7150 and 7250 in their environment.

 

[LodeRunner]

Wow, I wonder what caused that. Supply issues or critical design flaw?

 

[rootwyrm]

Wow, I wonder what caused that. Supply issues or critical design flaw?

It's stated in the emergency EOS document that it's supply shortages and unexpected discontinuations.
The ICX7150 and ICX7250 both use the same processor. Broadcom's been basically surprise-axing older silicon left, right, and center and moving last order dates up years. Everything below Trident2 already had a last order date, all of them got moved up to "too late."
Basically in the past 12 months, the entire Broadcom 1GbE portfolio was surprise axed.
BCM56344 is listed as 'active' with 52+ week lead time at supply houses. But Broadcom no longer lists it as an active part on their site and has scrubbed all references and all pages - meaning it has been surprise permanently discontinued.

 

[fohdeesha]

I think that's just End of Sale, not End of Life/End of Support.

yeah it's just end of sale due to supply chain, same reason juniper just end-of-sale-d their most popular MX and everything else based on HMC they can't get built anymore. end of support date for the 7250 is still 5 years out

 

[rootwyrm]

yeah it's just end of sale due to supply chain, same reason juniper just end-of-sale-d their most popular MX. end of support dare for the 7250 is still 5 years out

Read again. End of all software updates including bugfix and security is July 1, 2023 now, and allegedly Ruckus quit accepting all 4 hour hardware in February. The BCM56344's are 100% dead.

 

[fohdeesha]

Read again. End of all software updates including bugfix and security is July 1, 2023 now, and allegedly Ruckus quit accepting all 4 hour hardware in February. The BCM56344's are 100% dead.

I said end of support is 5 years out, which is what ruckus' own announcement says:

 

[rootwyrm]

I said end of support is 5 years out, which is what ruckus' own announcement says:

View attachment 22902

View attachment 22903

You might want to have someone read that sign to you.





Hopefully this is sufficiently clear.

 

[fohdeesha]

hey man that's really neat, but I said end of support, not end of software development. You alright? lmao

 

[fohdeesha]

Sheeit. All this time and I've just been freeloading! Donated!

Anyone think we'll start to see more 7250 models showing up on ebay now that they've been emergency suddenly EOL'd discontinued? Or do we think that folks will hang on to them since there's no current equivalent for a 24/48 port switch with > 4 sfp+ slots without going all the way up to the 7450. (Edited that for more accuracy).

Oh, and sorry to bump my own question, but does anyone have any ideas why I can't seem to get my 7250 to route properly?

thanks for the donation! as for your routing issues, you can ensure the switch itself is routing correctly by running "ping 192.168.0.254 source 192.168.10.254" - I would be really surprised if that doesn't work. next step would be having a PC in 192.168.0.0/24 with its gateway set to 192.168.0.254 try and ping 192.168.10.254 - if that works (it should), then the switch is working fine, I'd imagine the rest of your issues are related to your firewall having gateways/IP interfaces in all these vlans that the switch is trying to route but hard to say from glancing at your config

 

[thebwack]

ok, bought a 50M 12Fiber MPO cable on bay. I'm attaching a diagram it came with.

I also bought because I found one super cheap, a Systimax Instapatch panel thinking I could run a 50M MPO from our server room to the video suite and fan out from there.

I used a paper clip and flipped one end of one of the MPO Cables I have and plugged it in. if you look at the photo of the inside of the instapatch the first MPO input is going to Alpha side 1-12 which makes sense to me. With the flipped MPO though the lasers are lighting up fibers 1,2,3,4 and if I use the non-flipped as it came MPO cable it lights up 9,10,11,12

Looking at the back of the 6610 I'm trying the two right side 40GB ports (fan out ports) and I disabled stacking etc during setup. in the Web UI I see everything I have connected and I see those ports as 4 10GB ports.

If I plug a LC cable into 1,2 or 3,4 (or 9,10,11,12 with other cable) I get connected but no internet (in a windows system) from a solarflare card.

LC transmitters work fine in all the 10GB front ports, and I would have thought that the instapatch would light up 1,3,5,7 or 2,4,6,8 and give me 4 channels.

Is this instapatch made to do something else? I see there are different versions of it,. Any help is appreciated.

Oh, last question, should I just buy an MPO Breakout cable and coupler to go at other end?

EDIT: 9,10,11,12 with non flipped cable

 

[klui]

What are the Systimax and LC cable SKUs?

If your patch cables work for regular SFP+ ports then they are A-B cables, meaning TX on one end point to RX on the other. Because your MTP cable has method B polarity if things don't work probably your Systimax is method A. Then you will need to convert the LC cable to A-A. That's why it's important to know the specifications of the breakout cassette through its SKU. Good LC patch cords can have their polarity changed in the field but some can't.

Generally, to make things work with duplex fiber, there needs to be an odd number of polarity B cables in the chain.

Fiber Polarity Basics

Polarity defines direction of flow, such as the direction of a magnetic field or an electrical current. In fiber optics, polarity is directional; light signals travel through a fiber optic cable from one end to the other. A fiber optic link’s transmit signal (Tx) at end of the cable must match...

www.flukenetworks.com

Make sure you're not mixing OS2 cassette with OMx MTP trunk or vice versa. Your MTP appears to be OM4?