Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[dbvader]

Hope your (presumably vlan unaware) switch clients are not hooked up to tagged ports.

Perhaps run a traceroute from a client to 8.8.8.8 or some other internet IP address.

 

[Kirkenjerk]

Hope your (presumably vlan unaware) switch clients are not hooked up to tagged ports.

Perhaps run a traceroute from a client to 8.8.8.8 or some other internet IP address.

I got it working!!!! Edited my last comment but i'll add it here too. You pointed me in the right direction.

I needed to change that default-network and also realized I never setup the route for the specific VLAN I am testing (I was testing vlan 10 but then switched to VLAN 90 and never made that static route). I also had to make a change to the outbound NAT rule as the cidr mask wasnt correct.

I knew...just knew it was something stupid that I was doing. Thank you so much.

 

[richtj99]

Hi - would you mind reposting the working settings for your sh run? I would like to see how that works as my router is doing the routing but might be nice to have the brocade do it.

 

[Kirkenjerk]

Hi - would you mind reposting the working settings for your sh run? I would like to see how that works as my router is doing the routing but might be nice to have the brocade do it.

Current configuration:
!
ver 08.0.95mT213
!
stack unit 1
  module 1 icx7250-48-port-management-module
  module 2 icx7250-sfp-plus-8port-80g-module
  stack-port 1/2/1
  stack-port 1/2/3
!
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
 router-interface ve 1
 spanning-tree
!
vlan 10 name TenGig by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1
 untagged ethe 1/2/2
 router-interface ve 10
 spanning-tree
!
vlan 20 name DMZ by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 untagged ethe 1/1/24
 router-interface ve 20
 spanning-tree
!
vlan 30 name Security by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 untagged ethe 1/1/32
 router-interface ve 30
 spanning-tree
!
vlan 40 name Server by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 untagged ethe 1/1/40
 router-interface ve 40
 spanning-tree
!
vlan 50 name IOT by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 router-interface ve 50
 spanning-tree
!
vlan 60 name Home by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 untagged ethe 1/1/8
 router-interface ve 60
 spanning-tree
!
vlan 70 name OOB by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 untagged ethe 1/1/34
 router-interface ve 70
 spanning-tree
!
vlan 80 name Hosts by port
 tagged ethe 1/1/2 ethe 1/2/2
 untagged ethe 1/1/22 ethe 1/1/28 ethe 1/1/30
 router-interface ve 80
 spanning-tree
!
vlan 90 name Desktops by port
 tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
 untagged ethe 1/1/18
 router-interface ve 90
 spanning-tree
!
!
vlan 1000 name transit by port
 tagged ethe 1/2/8
 router-interface ve 1000
!
!
!
!
!
!
!
!
!
!
!
!
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
ip dhcp-client disable
ip default-network 10.0.0.0/8
ip default-network 10.0.0.1/32
ip route next-hop-enable-default
ip route 0.0.0.0/0 10.0.0.1
ip router-id 10.0.0.2
!
no telnet server
username super password .....
!
!
!
!
no web-management http
!
manager disable
!
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface ve 1
 ip address 10.1.1.254 255.255.255.0
!
interface ve 10
 ip address 10.10.10.1 255.255.255.0
!
interface ve 20
 ip address 10.1.2.1 255.255.255.0
!
interface ve 30
 ip address 10.1.3.1 255.255.255.0
!
interface ve 40
 ip address 10.1.4.1 255.255.255.0
!
interface ve 50
 ip address 10.1.5.1 255.255.255.0
!
interface ve 60
 ip address 10.1.6.1 255.255.255.0
!
interface ve 70
 ip address 10.1.7.1 255.255.255.0
!
interface ve 80
 ip address 10.1.8.1 255.255.255.0
!
interface ve 90
 ip address 10.1.9.1 255.255.255.0
!
interface ve 1000
 ip address 10.0.0.2 255.255.255.252
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
end

If you need the Opnsense settings let me know.

 

[TonyArrr]

I had a power question - I am thinking about upgrading a ICX6450 to a ICX7150-C12.

My POE usage on the 6450:

 Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
  1/1/2 On      On         12900      30000  802.3at  Class 4     3  n/a
  1/1/3 On      On          3800      30000  802.3at  Class 4     3  n/a
1/1/15 On      On          2300      15400  802.3af  n/a         3  n/a
1/1/18 On      On          2900      15400  802.3af  n/a         3  n/a
1/1/19 On      On          4300      15400  802.3af  n/a         3  n/a
1/1/39 On      On          2500      15400  802.3af  n/a         3  n/a
--------------------------------------------------------------------------
Total                     28700     121600

The ICX-7150-C12 has a budget of 124000 mWatts.

The consumed power on the 6450 is 28700 mWatts & allocated is 121600 mWatts which should be under the POE budget & OK

BUT
before i make a switch i wanted to make sure this would work.

ICX7150 - 10 watts idle
ICX6450 - 50 watts idle (48 port)

Savings of 40 watts - idle to idle

Looks like it is a 3 year breakeven on the cost of electric.

Thanks,
Rich

My only thought here is to make sure there is at least a little airflow running by the 7150-C12. Mine has PoE off and it still gets quite toasty. When I did my read through of this thread, there was a patch where people kept having the PSUs of these suffer heat death (very fixable, but obviously annoying).

It doesn’t take much to keep it very cool though, a 80mm fan sitting on top at 900rpm has it as cool as a consumer dumb switch and it’s totally inaudible still.

None of this is to say they’re bad switches, they just seem to be walking a fine line with the heat by the time they make it to the second hand market. They’re otherwise great, I’m actually on the prowl for a second one

 

[Sealside]

Thank you. I will check, maybe I will try to remove the 40x40 fans completely and put 2 times 80x80 in the top of the case. They will run more silently with even more throughput.

I'm in the same situation. I have 3x 40mm mechatronics in the back and a 20mm on the asic cooler. Still after some time they will spin up. My conclusion is to go for 80mm fans in order to reach acceptable noise and cooling.

Another approach would be to take noctuas 40mm and always run them on max voltage plus faking rpm with a esp8266, but i don't think the will provide enough cooling for the long run.

/S

 

[junicast]

I'm in the same situation. I have 3x 40mm mechatronics in the back and a 20mm on the asic cooler. Still after some time they will spin up. My conclusion is to go for 80mm fans in order to reach acceptable noise and cooling.

Another approach would be to take noctuas 40mm and always run them on max voltage plus faking rpm with a esp8266, but i don't think the will provide enough cooling for the long run.

/S

I tried just that. Actually I went for 2 x 120mm fans but when the system slow the fans down, the switch is only reachable for like 1 or 2 seconds and then suddenly reboots. The same happens if I connect 2 x 80mm fans.
I do NOT know but my guess is there is a threshold in the firmware that the fans must have a minimum speed and if that's not guaranteed, the system reboots in order to *fix* the *problem*.

Edit:
I tried with one Maglev 40mm fan combined with a 120mm case fan. This does not result in a rebooting device. This is the fan status.

SSH@7250.example.com#dm fan

Fan 1 Speed at 573 RPM.

Fan 2 Speed at 2760 RPM.

 

[junicast]

I wrote my own little thread about my experiences with the ICX 7250.

 

[fohdeesha]

yes the site's down looks like the mobo might have died on the box this is on, it might be a while until I can get remote hands to check it out so I'll spin up the vm in another location sometime today

 

[donut_taganes]

I have been browsing the docs using the wayback machine - Fohdeesha Docs

shitty ninja edit - thank you fohdeesha for all your work here making affordable 10G at home a lot easier.

 

[XPEric]

Does anyone happen to have the "brocade-12-19-2023.zip" that was previously available for download from the website in OP's post? The site seems to be down, and while the Internet Archive has a copy of the text of the instructions, it doesn't have the file. I do apologize if this was resolved in a previous reply, but I can't seem to find anything.

Thanks,
Eric

 

[R3Z3N]

Yes, I almost always download directions...because I have a tendency to bring down my homelab just to try stuff....my better half may be unhappy, but she has no need for any networking at the house as she is perfectly fine on 5G. PM'd you the zip btw.

Anyway, got my switch working fine from a few pages back. Oddly POE hasn't been a problem starting up the past 2 reboots. Now to figure out DHCP via PFSense and Routing on the 7250...but alas the Brocade is non-authoritative and PFSense well...can't do DHCP outside of the subnets one has assigned. Hmm, wanted to go to VyOS but alas no VPP there anymore either. Ah bugger Broadcom TW*TS! May end up just paying for TNSR then.

 

[fohdeesha]

apologies, the site has been up on a new server since this morning but namecheap seems to have their own definition of "updating your ns glue records". I'm calling them now to get this fixed and probably moving registrars because this is not the first time I've had issues setting glue records with them. in the meantime you can get to the site by sticking this in your hosts file

23.139.82.211 fohdeesha.com

EDIT: it's finally updated, should be reachable now

 

[blunden]

Hmm, wanted to go to VyOS but alas no VPP there anymore either. Ah bugger Broadcom TW*TS! May end up just paying for TNSR then.

VPP for VyOS is only available as a plugin for which they haven't decided if it's going to be a paid feature or not yet.

There is a homelab license option for TNSR, unless they've removed that again. You never know with Netgate. There can be an issue with updating kernel version when on the free version (on certain hardware) though since you don't have access to certain packages from the paid repository.

TNSR Home+Lab - apt update/upgrade safety

Hello, With TNSR Home+Lab, is it safe to run: apt update apt upgrade i.e. Will running the above have any impact on the TNSR software? Thanks, Mike

forum.netgate.com

 

[R3Z3N]

No more TNSR homelab....

 

[blunden]

No more TNSR homelab....

Ah yes, that seems to be true. Seems to have happened right after I last looked into it.

https://www.reddit.com/r/Netgate/comments/17llr2z
It seems somewhat likely that it will become a paid feature for VyOS too, but perhaps with a pricing model more palpable for home lab use. Nothing has been decided yet though as far as I know.

 

[sergi0]

I think you would need to undo the tagging and dual modes on 15 and 17 first.

I migrate the 6450-24 toward a 7150-24 as it is fanless (everything is in a closet so I am hunting for lower wattage when I can). I try to put my vlan conf that you help build, but it seems that in 7150, the dual-mode option is deprecated. I read a couple of docs and understand that I need to do something differently but so far no luc.
Any ideas ?

 

[R3Z3N]

Yes no dual mode needed anymore. Just use untagged and tagged vlans. Of course only 1 untagged per port.

I recommend downloading the manuals for the Fastiron os you are using. Even as a CLI noob( however IT repair for decades) the rather large manuals have made things easy to follow, along with Terry's videos. The hard thing about the manuals is they don't explain concepts, but this is why I want to take a night class even 20 years after college.

 

[sergi0]

Yes no dual mode needed anymore. Just use untagged and tagged vlans. Of course only 1 untagged per port.

I recommend downloading the manuals for the Fastiron os you are using. Even as a CLI noob( however IT repair for decades) the rather large manuals have made things easy to follow, along with Terry's videos. The hard thing about the manuals is they don't explain concepts, but this is why I want to take a night class even 20 years after college.

I just found out Terry video and I will download the manual. I was looking for the dual-mode option and finally discover it was not needed anymore.
The setup is now complete, I will probably migrate the switch tonight or tomorrow morning .
Thanks for all the helps I got on this forum.

 

[MacOS_Guy]

I currently have (2) ICX-7250's and (1) ICX-6650. The 6650 is mainly to hold VLAN's for 10gb networks on different subnets with no LAN or Internet connection. Before I open this up and try to modify the fans in some way shape or form (similar to what some guy did for a 6610 maybe), does anyone know of a 48 port 10gb SFP+ switch thats maybe a little more modifiable or quieter? Everything going to this switch is fiber.