Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[OKGolombRuler]

Does the ICX have a route to 192.168.101.1? What is in the IPv4 routing table on the ICX?

Yes, and it pings fine.

ping 192.168.101.1
Sending 1, 16-byte ICMP Echo to 192.168.101.1, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 192.168.101.1   : bytes=16 time=1ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms.    

traceroute 192.168.101.1

Type Control-c to abort
Tracing the route to IP node 192.168.101.1(192.168.101.1) from 1 to 30 hops

  1     1 ms   <1 ms   <1 ms 192.168.101.1

The ICX's routes:

sh ip route                                                                                                                
Total number of IP routes: 9                                                                                                                          
Type Codes - B:BGP D:Connected O:OSPF R:RIP S:Static; Cost - Dist/Metric                                                                              
BGP  Codes - i:iBGP e:eBGP                                                                                                                            
OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2                                                                                        
STATIC Codes - v:Inter-VRF                                                                                                                            
        Destination        Gateway         Port          Cost          Type Uptime                                                                    
1       0.0.0.0/0          192.168.99.227  ve 99         110/2         O1   2d8h                                                                      
...
6       192.168.9.0/24     DIRECT          ve 9          0/0           D    2d8h                                                                      
...                                            
9       192.168.101.1/32   192.168.99.227  ve 99         110/2         O    2d3h

and the MT's routes:

 /ip route print
Flags: D - DYNAMIC; A - ACTIVE; c, s, o, d, y - COPY
Columns: DST-ADDRESS, GATEWAY, DISTANCE
#     DST-ADDRESS       GATEWAY             DISTANCE
  DAd 0.0.0.0/0         MyISPSubnet.1                1
  DAc MyISPSubnet.0/24    outside               0
0  As 192.168.0.0/16    192.168.99.254             1
...
DAo 192.168.9.0/24    192.168.99.254%v99       110
...
  DAc 192.168.101.1/32  loopback                   0

 

[Mark]

Just tried a 100G QSFP28 to 4X 25G SFP28 Breakout DAC in one of the 100g ports on the 7650 and it does not appear to be supported as I'm getting the following from show media: Port 1/3/2: Type : 40GBASE-Passive Copper 1m (QSFP+) (Not Supported)

The cable is juniper coded any idea if this would have anything to do with it? Doesn't seem like there is anything documented around breaking out the 100g port on the 7650.

 

[itronin]

Just tried a 100G QSFP28 to 4X 25G SFP28 Breakout DAC in one of the 100g ports on the 7650 and it does not appear to be supported as I'm getting the following from show media: Port 1/3/2: Type : 40GBASE-Passive Copper 1m (QSFP+) (Not Supported)

The cable is juniper coded any idea if this would have anything to do with it? Doesn't seem like there is anything documented around breaking out the 100g port on the 7650.

hmmm. I just picked one up off the bay (need a new house switch since we are moving). I did some brief reserarch and did not see anything about 4x breakout on the 100's I thought they could only run 100 or 40. The 40 ports I thought were also not breakout. I also read that if you use the rear ports for uplink instead of stacking then it disables the front modular bay. Actually it literally says "disables configuration" of the modular bay ports. Not sure if that means they will or won't work. Hoping to get mine on the bench this coming weekend to play with it.

Honestly though I'm more interested in the 24 ports of multi-gig on mine. Probably still use my icx6610's for sfp+ ports in the rack.

 

[MESSERCHMIDT]

do any of these units support 2.5 or 5gbs? or just 1gbs/10gbs? (and some 40gbs on the back) [specifically the Brocade ICX6610]

 

[TonyArrr]

do any of these units support 2.5 or 5gbs? or just 1gbs/10gbs? (and some 40gbs on the back) [specifically the Brocade ICX6610]

I think I saw one of the 7150 or 7450 24 port models did some 2.5g, but NbaseT is a bit of a bastardised standard built to save a few bucks over 10GbaseT when making consumer gear, so a lot of enterprise level gear just doeouch it, taking a position of “if it’s worth going over 1G, why would we not do 10?”

There is a new line coming out literally right now, the 8200 line, which the 12-port one has all their downlink ports as 2.5s to support running wifi 6 APs, but the very little I’ve been able to find on pricing suggests a cool 2k US for them when they get to sale

I remember reading a thread where someone was asking how to get a 2.5g SFP module to work in a 6610 and the Ruccus support rep just told them to use 10g modules, and they didn’t plan to write support into the ICX software base because there was no reason in the target market to do 2.5 anyway.

Old thread, definitely before wifi 6, so things may have changed since then. Probably no new firmware for something like the 6610 to run it properly, but @fohdeesha would know much better than I on that front

 

[Mark]

hmmm. I just picked one up off the bay (need a new house switch since we are moving). I did some brief reserarch and did not see anything about 4x breakout on the 100's I thought they could only run 100 or 40. The 40 ports I thought were also not breakout. I also read that if you use the rear ports for uplink instead of stacking then it disables the front modular bay. Actually it literally says "disables configuration" of the modular bay ports. Not sure if that means they will or won't work. Hoping to get mine on the bench this coming weekend to play with it.

Honestly though I'm more interested in the 24 ports of multi-gig on mine. Probably still use my icx6610's for sfp+ ports in the rack.

I finally found a document that shows what cables they support and yeah its a no go on breaking out those ports for the time being at least. It appears that they have been adding support for additional cables in the later software releases. PDF with details below:
https://www.commscope.com/globalassets/digizuite/61722-ds-ethernet-optics-family.pdf

I didn't actually buy the switch specifically for breaking out the 100g ports but I figured if I could run 2x25g to each of my 3 esxi servers that would save me some of the multi gig ports in the front. I've got the switch racked up and its really amazing I have 5 Cisco 4800 series AP's that do 5g multi gig and a bunch of 10g workstations. I still need to setup a stack to the 7150 in my attic. I'll keep my fingers crossed that they add breakout support in the future!

 

[Scarlet]

Just wanted to add my 2 cents on how much power ICX6450-24P (PoE models) draw when idle and with nothing but power connected.

I tested three ICX6450-24P (using a Shelly Plus 1PM), and all three consistently reported the same values:

25 W power draw during boot (when the two fans run at full speed)
22 W power draw after boot (when the two fans drop to low speed)
35 W power draw after a few seconds when the PoE power supply is initialized

So it seems the PoE power supply adds ~13 W on top of the idle power without PoE.

 

[DarthCirrus]

Hello!

This is a great thread, lot of great information on the first page! But I don’t see any 24 or 48 port 10g RJ45 switches, preferably with 4 or more QSFP+ 40g or QSFP28 100g.
I see one 48 port SFP+, but the cost of those MicroTik adaptors seems to make it prohibitively expensive for a big 10+ station 10GbaseT deploy.

Any recommendations in the Brocade family for switches that would meet those recommendations?

 

[bitbckt]

That's the ICX 7650 series, if you can find one. ICX 7650-48ZP for example. I'd prefer an Arista 7050T that might actually be found on the market, personally.

 

[DarthCirrus]

That's the ICX 7650 series, if you can find one. ICX 7650-48ZP for example. I'd prefer an Arista 7050T that might actually be found on the market, personally.

Thanks for the quick response!

Wow! Those Brocade switches are not readily available on eBay and very expensive; I can see why you recommend the Arista 7050T

 

[Matta]

Need help with Brocade ICX6610-48p:
Currently I have Default VLAN ID:1 with all ports untagged.
Through web gui, how to add two VLANs (VLAN ID:100 AND VLAN ID:200) to all ports, but to also keep default untagged ports also, for my main network ?
Thanks !
EDIT:
For example - If I try to add some ports to VLAN ID:100, they get deducted from VLAN ID:1.

 

[Vesalius]

Need help with Brocade ICX6610-48p:
Currently I have Default VLAN ID:1 with all ports untagged.
Through web gui, how to add two VLANs (VLAN ID:100 AND VLAN ID:200) to all ports, but to also keep default untagged ports also, for my main network ?
Thanks !
EDIT:
For example - If I try to add some ports to VLAN ID:100, they get deducted from VLAN ID:1.

You are looking for dual-mode. Doable from the web-gui, but if memory serves convoluted and not intuitive.

Commscope Technical Content Portal

docs.commscope.com

 

[Matta]

You are looking for dual-mode. Doable from the web-gui, but if memory serves convoluted and not intuitive.

Commscope Technical Content Portal

docs.commscope.com

If I add port (that's connected to my main router) to VLAN and make it dual mode, I lose connection on it.

EDIT:
I've managed to configure it successfully by using terminal. I haven't figured out how to do it via web gui.

 

[funky]

Just had the strangest thing happen on my FCX648S, wanted to know if anyone had seen this kind of behavior...

• Switch had no lights at all on the ports, links were down on the remote side
• Fans were running as per usual (can't tell you if they were loud or quiet, on this switch they have one noise level - loud)
• POE was still powering end devices
• A pull of the power cable fixed it, all came back up
• Except one piece - a 'dual-mode NNNN' statement was dropped from a single interface, but not other interfaces
• The other interfaces had been added after the one that had the statement dropped, so the dropped statement had for sure been written
• I didn't attempt to get anything off the console, just got that side of the house back up and running

Anyone had similar behavior?

 

[gseeley]

Has anyone upgraded (ICX 7xxx) to 09.0.10d yet?

 

[selta]

If I add port (that's connected to my main router) to VLAN and make it dual mode, I lose connection on it.

EDIT:
I've managed to configure it successfully by using terminal. I haven't figured out how to do it via web gui.

Just avoid the web GUI. It'll save you a lot of headache

 

[TonyArrr]

Anyone have recommendations for SFP+ modules for the 7250 and 7150 switches? Like, any that work real well, or any that really really don’t?
Mostly connecting to OM5, though will probably have 3x 50cm DACs too

 

[LodeRunner]

Anyone have recommendations for SFP+ modules for the 7250 and 7150 switches? Like, any that work real well, or any that really really don’t?
Mostly connecting to OM5, though will probably have 3x 50cm DACs too

Anything from fs.com would be fine. The switches aren't picky according to other people in this thread; but my only experience is with fs.com modules. I order them coded as Brocade. Cisco generic coding is also fine from what I've read. I have their SFP/QSFP code box so I code the ends of DACs to match the devices, but the cables technically worked fine without that (I have a mix of Mellanox, Brocade/Ruckus, Arista).

 

[TonyArrr]

Anything from fs.com would be fine. The switches aren't picky according to other people in this thread; but my only experience is with fs.com modules. I order them coded as Brocade. Cisco generic coding is also fine from what I've read. I have their SFP/QSFP code box so I code the ends of DACs to match the devices, but the cables technically worked fine without that (I have a mix of Mellanox, Brocade/Ruckus, Arista).

Sweet, I’ll go and pick some from there. I am working my way through this thread for info, but only up to page 170 so far and one of my new switches will be here tomorrow

thanks for the info!

 

[OKGolombRuler]

@kpfleming -- the stupid idea fairy visited me. Think I could fake having a VRF with dot1q (qinq) tunneling? it would abstract away the "real"/customer VLANs from the switch port I want to bring them "out of". I'd need to figure out how to set route preferences to use the FW path if it's up; I'm not sure if I can get e.g. OSPF to prefer a lower-cost OSPF route over a directly-connected route. (Which seems both stupid, and counterintuitively useful...)
Thoughts?