Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

sash

Member
Nov 22, 2019
31
4
8
...If you REALLY don't want this address having internet access, then learn to specify the ICX source address in commands requiring internet access so it doesn't default to using 10.0.0.1, and choose an ICX source address that does have a return route. example: "ping 8.8.8.8 source 192.168.0.11" - like magic, your ICX will have internet access. Same with other protocols on the switch, for instance NTP can be told which source address to use with "source-interface ve 10"

This is not a NAT issue, it's a routing issue, your edge router does not have return routes for a handful of your ICX addresses. The addresses that do have return routes do have internet access, the addresses that do not have return routes do not have internet access. I'm not sure how else to make this any clearer besides painting pictures
Appreciate your input. I will have to reconfigure NTP to use source address of virtual interface that is configured with OSPF routing then. That seems to be the solution in my case as I do not want 10.0.0.x subnet to have internet access. It is still strange to me why ICX would default to using lowest subnet configure for its own traffic, i.e. NTP and not use directly connected IP 192.168.29.1 as source.
 
  • Like
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,084
1,943
113
30
fohdeesha.com
Appreciate your input. I will have to reconfigure NTP to use source address of virtual interface that is configured with OSPF routing then. That seems to be the solution in my case as I do not want 10.0.0.x subnet to have internet access. It is still strange to me why ICX would default to using lowest subnet configure for its own traffic, i.e. NTP and not use directly connected IP 192.168.29.1 as source.
Routers/l3 switches typically get configured with a lot of IP addresses in practice, so it's been commom across vendors for some time now that when left to guess, it picks the lowest IP address to use, so there is at least some convention to go by (versus just picking source addresses at random). The switch doesn't really have the same understanding of "direct connected", in a lot of other configurations this would mean something totally different. This is all why most commands allow you to override the fallback source IP address and specify your own
 

ajax3712

New Member
May 22, 2018
7
2
3
Currently have a ICX-7250-48p (much thanks to @fohdeesha for the info & guides!). Still settings things up and still learning, but I am excited to play around. I have what may be a silly question... Is it possible or even advisable to do mixed stacking with the 7250 with other models (e.g. 7150)?

When I searched the forums for "mixed stack", I only saw the 6610/6450 come up as results. Brocade docs only discuss stacking with other 7250s...
 
  • Like
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,084
1,943
113
30
fohdeesha.com
Currently have a ICX-7250-48p (much thanks to @fohdeesha for the info & guides!). Still settings things up and still learning, but I am excited to play around. I have what may be a silly question... Is it possible or even advisable to do mixed stacking with the 7250 with other models (e.g. 7150)?

When I searched the forums for "mixed stack", I only saw the 6610/6450 come up as results. Brocade docs only discuss stacking with other 7250s...
no mixed model stacking on the 7250 (or any icx7xxx models that I know of)
 

SuperServer

New Member
Feb 1, 2019
15
1
3
Anyone have a spare poe board to part with? Was hoping not to pay for a full switch just to get a poe board from it.
 

klui

Active Member
Feb 3, 2019
275
118
43
If you look at the 6610 fan trays, each tray has 2x fans with a bit of space between them, but they flow in the same direction. Not counter rotating so to speak, and not joined together. The fan connector for the tray has enough wires for the two fans. So...

Took two trays...took off one fan from each tray, wired the second tray to the first one (extend the wires enough to go around the stacking card) and put them back in. It is...subjectively quieter than two fan trays (with 4x fans vs 2x fans in two trays) with slightly less power draw.
Interesting.

If the trays are connected to each other, how do you remove them? Do you have the wires long enough so one tray can be removed and then you disconnect the headers?
 

kapone

Well-Known Member
May 23, 2015
950
531
93
Interesting.

If the trays are connected to each other, how do you remove them? Do you have the wires long enough so one tray can be removed and then you disconnect the headers?
You don't.

If you're gonna **ck with the internals of the switch, you can't expect things to remain the same. :)
 

cserve

New Member
Feb 17, 2019
2
0
1
I'm not sure if that's possible, but given that very specific error message, it looks like it's not. I'm running a bit of a mix of that and the 40g ports on my home stack, it's stacked via a trunk of 1 40g port and 1 breakout port, leaving me 2x 40gbE ports and 2x breakout ports

it's also worth noting stack traffic over the 4x 10gbE QSFP+ ports is hashed similar to standard LACP, eg by source IP / port. Meaning if you stack using only the breakout ports, you won't get single-stream speed above 10gbps between stuff connected to each stack member. so 40gbE host on switch 1 is not going to be able to do more than 10gbps for single stream stuff (eg something using only one source IP and one source port, like SMB shares etc (unless running SMB 3.0)) to a 40gbE device on switch 2, as it will get hashed to just 1 of the 10gbps stacking channels
Seems like it is not possible to stack two 6610 using only the breakout ports. That is a bummer.

I was hoping I can get 4x 40Gbe out of these and not have to add another 40Gbe switch due to cost/power consumption.

The next plan is to unstack them and set up LACP (6610 does not support MCT sadly) using the breakout ports only. Still limit by 10Gbps per single stream as you mentioned but that does not affect my use case much, only need total aggregated bandwidth between nodes.

Hope this work. Is there any downside to unstacking beside ease of management? Redundancy is not a high priority since this is home lab.
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,084
1,943
113
30
fohdeesha.com
Seems like it is not possible to stack two 6610 using only the breakout ports. That is a bummer.

I was hoping I can get 4x 40Gbe out of these and not have to add another 40Gbe switch due to cost/power consumption.

The next plan is to unstack them and set up LACP (6610 does not support MCT sadly) using the breakout ports only. Still limit by 10Gbps per single stream as you mentioned but that does not affect my use case much, only need total aggregated bandwidth between nodes.

Hope this work. Is there any downside to unstacking beside ease of management? Redundancy is not a high priority since this is home lab.
that should work (running lacp), not many downsides for home use, you just lose the "single device to manage" as you point out
 

TechGambino

New Member
Jan 25, 2019
4
0
1
I saw earlier in the thread there may have been an issue with passive DAC cables on the breakout ports. Was this confirmed?
Just in case someone stumbles in here with this question again, I had no problem with passive DAC cables from fs.com and the 8x 10gb breakout ports worked fine without a license.

Thanks for the getting started guide @fohdeesha
 

NYCone

Member
Jun 23, 2017
35
8
8
58
I just set up my ICX 6610 following fohdeesha's awesome guide.

It seems to be working perfectly except I'm a little confused about vlans.

I'm trying to set up multiple tagged vlans on a "trunk" with vlan ID 1 being untagged.

I watched "the death of dual-mode" video:

which says dual mode has been depreciated, but my firmware doesn't seem to be following that guideline. I'm updated to 08030t.

I can get dual-mode up on the port, but I'm curious about the new firmware death of dual mode.