Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

TeleFragger

Active Member
Oct 26, 2016
263
55
28
51
Xeon D-1541 X10SDV-8C-TLN4F w Intel XXV710-DA2 PFSense iperf server
Atom C3758 A2SDi-8C-HLN4F w Intel X550-T2 (getting changed to X710-DA2) Proxmox w VMs paused except for debian networking test os
AMD 5950x w Intel XXV710-DA2 Windows, but moving Windows over to VM soon as well on this HW.
gotcha.. i have been chasing windows issues on x520-da2 only getting me 2.5gb but boot machines into parted magic and get 9.4gb...
 

junicast

New Member
Mar 16, 2024
13
1
3
You can splice into one of the existing fans. I'd recommend using new connectors instead of trying to splice the wires themselves.
Thank you. I will check, maybe I will try to remove the 40x40 fans completely and put 2 times 80x80 in the top of the case. They will run more silently with even more throughput.
 

BoGs

New Member
Feb 18, 2019
29
4
3
For the ICX6610 I need some more 10G SFP ports then the front and for the breakout ports on the back (right) seems simple LR4 and OS2 cable harness.

Is that possible for the two left QSFP+ ports? or are those dedicated to 40g only link? even if using an LR4 (or SR4) optic?
 

Matta

Member
Oct 16, 2022
30
4
8
I was wondering, what kind of CPU Brocade ICX6610 has ? I know it's some sort of PowerPC processor.
And how does it compare to, for example, Mikrotik CCR2116-12G-4S+ processor (Annapurna Labs Alpine AL73400 , 16 cores @ 2000Mhz).
Are L3 capabilities same, lower or higher than this particular Mikrotik ? I'm aware that one is switch and other is fully fledged router but still, one is enterprise gear, other is "affordable" class.
 

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,345
820
113
I was wondering, what kind of CPU Brocade ICX6610 has ? I know it's some sort of PowerPC processor.
And how does it compare to, for example, Mikrotik CCR2116-12G-4S+ processor (Annapurna Labs Alpine AL73400 , 16 cores @ 2000Mhz).
Are L3 capabilities same, lower or higher than this particular Mikrotik ? I'm aware that one is switch and other is fully fledged router but still, one is enterprise gear, other is "affordable" class.
The CPU on the ICX6610 doesn't do a lot. It's purely management plane, and only very specific kinds of traffic are processed on the CPU.

All routing and switching happens inside the ASICs at wirespeed.
 

Matta

Member
Oct 16, 2022
30
4
8
The CPU on the ICX6610 doesn't do a lot. It's purely management plane, and only very specific kinds of traffic are processed on the CPU.

All routing and switching happens inside the ASICs at wirespeed.
Thanks for the clarification.
However, as all layer 3 switches, they are - after all - switches. Proper router is still needed if there's plan to do anything beyond plain simple routing (NAT, firewall, etc.).
 

i386

Well-Known Member
Mar 18, 2016
4,245
1,546
113
34
Germany
Proper router is still needed if there's plan to do anything beyond plain simple routing (NAT, firewall, etc.).
bgp, ospf and rip v1 & 2 are "proper" routing protocols and supported by the icx 6610, 6450, 7250 etc....
Nat, firewall etc are functions that you find on devices that cisco calls "isr" (integrated services router)
 
  • Like
Reactions: Jason Antes

richtj99

Member
Jul 8, 2017
70
1
8
51
I had a power question - I am thinking about upgrading a ICX6450 to a ICX7150-C12.

My POE usage on the 6450:

Code:
 Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
  1/1/2 On      On         12900      30000  802.3at  Class 4     3  n/a
  1/1/3 On      On          3800      30000  802.3at  Class 4     3  n/a
1/1/15 On      On          2300      15400  802.3af  n/a         3  n/a
1/1/18 On      On          2900      15400  802.3af  n/a         3  n/a
1/1/19 On      On          4300      15400  802.3af  n/a         3  n/a
1/1/39 On      On          2500      15400  802.3af  n/a         3  n/a
--------------------------------------------------------------------------
Total                     28700     121600
The ICX-7150-C12 has a budget of 124000 mWatts.

The consumed power on the 6450 is 28700 mWatts & allocated is 121600 mWatts which should be under the POE budget & OK

BUT
before i make a switch i wanted to make sure this would work.

ICX7150 - 10 watts idle
ICX6450 - 50 watts idle (48 port)

Savings of 40 watts - idle to idle

Looks like it is a 3 year breakeven on the cost of electric.

Thanks,
Rich
 

Kirkenjerk

New Member
Dec 19, 2022
7
1
3
Really hoping someone can shed some light on what I am doing wrong. I've been trying to figure this out for like 2 weeks now and still can't figure out how to get my setup working. I am trying to improve my networking skills so don't hate me too much if I've done something stupid.

Firewall: Opnsense
L3/Router: ICX 7250

Needs:
ICX 7250 handles all routing for the network, and the Opnsense box only does firewall things.

Problems:
I can get inter-vlan routing working (somewhat), but keep running into issues where the VLANs on the 7250 cannot hit the internet. I am not sure if its a routing issue on the switch or a NAT/Firewall/Routing issue on the Opnsense box. I can hit the Opnsense IP, and WAN IP, but not anything beyond that.

I also think I broke my config so now I can't get anything working at the moment.


If anyone has this same setup and can provide info on their Opnsense settings and their ICX 7250 config that would be amazing!

-------------------------------------------------------------------------------------------------------------------------------------------------------
Opnsense Settings:

Transit VLAN (VLAN tag 1000) for communication with switch (10.0.0.1/30) - Firewall has an IP of 10.0.0.1 on this VLAN, and the Switch has an IP of 10.0.0.2 on this VLAN. This VLAN is assigned to the ix0 port on my opnsense box, and I've tagged this VLAN on port 1/2/8 on the switch (and I've also untagged it to no avail).

Gateway was created on Opnsense for this Transit VLAN, the Gateway IP is 10.0.0.2 (the IP of the transit vlan on the switch).
1711299253298.png
Static Routes for each VLAN that exists on the switch, that is, 10.1.X.0/24 which use the Transit Gateway (10.0.0.2).

Firewall rules for the Transit VLAN on the opnsense are set to allow anything out and in. No restrictions in place here.
1711299240043.png

Outbound NAT rules are configured to allow anything outbound for each VLAN on the switch.
1711299216305.png

ICX 7250 Running Config:

Code:
Current configuration:
!
ver 08.0.95mT213
!
stack unit 1
  module 1 icx7250-48-port-management-module
  module 2 icx7250-sfp-plus-8port-80g-module
  stack-port 1/2/1
  stack-port 1/2/3
!
!
global-stp
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
spanning-tree
!
vlan 10 name TenGig by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1
untagged ethe 1/2/2
router-interface ve 10
spanning-tree
!
vlan 20 name DMZ by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
untagged ethe 1/1/24
router-interface ve 20
spanning-tree
!
vlan 30 name Security by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
untagged ethe 1/1/32
router-interface ve 30
spanning-tree
!
vlan 40 name Server by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
untagged ethe 1/1/40
router-interface ve 40
spanning-tree
!
vlan 50 name IOT by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
router-interface ve 50
spanning-tree
!
vlan 60 name Home by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
untagged ethe 1/1/8
router-interface ve 60
spanning-tree
!
vlan 70 name OOB by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
untagged ethe 1/1/34
router-interface ve 70
spanning-tree
!
vlan 80 name Hosts by port
tagged ethe 1/1/2 ethe 1/2/2
untagged ethe 1/1/22 ethe 1/1/28 ethe 1/1/30
router-interface ve 80
spanning-tree
!
vlan 90 name Desktops by port
tagged ethe 1/1/2 ethe 1/1/22 ethe 1/1/28 ethe 1/1/30 ethe 1/2/1 to 1/2/2
untagged ethe 1/1/18
router-interface ve 90
spanning-tree
!
!
vlan 1000 name transit by port
tagged ethe 1/2/8
router-interface ve 1000
!
!
!
!
!
!
!
!
!
!
!
!
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
ip dhcp-client disable
ip default-network 10.0.0.2/32
ip route next-hop-enable-default
ip route 0.0.0.0/0 10.0.0.1
ip router-id 10.0.0.2
!
no telnet server
username super password .....
!
!
!
!
no web-management http
!
manager disable
!
!
manager port-list 987
!
!
!
!
!
!
!
!
!
interface ve 1
ip address 10.1.1.254 255.255.255.0
!
interface ve 10
ip address 10.10.10.1 255.255.255.0
!
interface ve 20
ip address 10.1.2.1 255.255.255.0
!
interface ve 30
ip address 10.1.3.1 255.255.255.0
!
interface ve 40
ip address 10.1.4.1 255.255.255.0
!
interface ve 50
ip address 10.1.5.1 255.255.255.0
!
interface ve 60
ip address 10.1.6.1 255.255.255.0
!
interface ve 70
ip address 10.1.7.1 255.255.255.0
!
interface ve 80
ip address 10.1.8.1 255.255.255.0
!
interface ve 90
ip address 10.1.9.1 255.255.255.0
!
interface ve 1000
ip address 10.0.0.2 255.255.255.252
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
end
If there is any info I am missing please let me know!
 
Last edited:

richtj99

Member
Jul 8, 2017
70
1
8
51
Code:
Current configuration:

optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
ip dhcp-client disable
ip default-network 10.0.0.2/32
ip route next-hop-enable-default
ip route 0.0.0.0/0 10.0.0.1
ip router-id 10.0.0.2
Im not sure but my config has

ip dns server-address 10.0.0.2

Maybe pointing the dns to 10.0.0.2 - your brocade or 10.0.0.1 - your firewall - or even 8.8.8.8 for testing?
 

Kirkenjerk

New Member
Dec 19, 2022
7
1
3
Im not sure but my config has

ip dns server-address 10.0.0.2

Maybe pointing the dns to 10.0.0.2 - your brocade or 10.0.0.1 - your firewall - or even 8.8.8.8 for testing?
I should have added, that from VLAN 1 on the switch (IP of 10.1.1.254) I can ping the IP of the Opnsense firewall (which is 10.1.1.1), and the internet without issue. It's only the VLANs on the 7250 that cant ping or reach the internet at all.

I don't have my DNS servers or DHCP servers configured on the 7250 at the moment, everything is done via IP or set statically until I can figure out the routing/firewall issues.
 

dbvader

New Member
Oct 22, 2023
15
1
3
everything is done via IP or set statically until I can figure out the routing/firewall issues.
Do the clients connected to the switch have 10.0.0.1 or 10.0.0.2 as their (default) gateway? I believe it should be 10.0.0.2 with your config. If the gateway on the clients is 10.0.0.1 then I don't see how they would be able to reach that IP (without intermediate routing via 10.0.0.2).

opnsense (10.0.0.1) <-> switch (10.0.0.2) <-> client
 

Kirkenjerk

New Member
Dec 19, 2022
7
1
3
Do the clients connected to the switch have 10.0.0.1 or 10.0.0.2 as their (default) gateway? I believe it should be 10.0.0.2 with your config. If the gateway on the clients is 10.0.0.1 then I don't see how they would be able to reach that IP (without intermediate routing via 10.0.0.2).

opnsense (10.0.0.1) <-> switch (10.0.0.2) <-> client
When setting up the VE for each vlan I set an IP. The IP of the VE is the gateway for each vlan.

I don’t think 10.0.0.2 should be set as the gateway for each client.
 

dbvader

New Member
Oct 22, 2023
15
1
3
Sort of what I meant, albeit for the wrong vlan (1000) which is clearly your opnsense <-> switch vlan.

Your switch's default route seems to be pointing to itself (10.0.0.2) instead of the opnsense IP (10.0.0.1).

Code:
ip default-network 10.0.0.2/32
 

Kirkenjerk

New Member
Dec 19, 2022
7
1
3
Sort of what I meant, albeit for the wrong vlan (1000) which is clearly your opnsense <-> switch vlan.

Your switch's default route seems to be pointing to itself (10.0.0.2) instead of the opnsense IP (10.0.0.1).

Code:
ip default-network 10.0.0.2/32
You are absolutely right. I’ll try changing that and see what happens.
 

Kirkenjerk

New Member
Dec 19, 2022
7
1
3
Sort of what I meant, albeit for the wrong vlan (1000) which is clearly your opnsense <-> switch vlan.

Your switch's default route seems to be pointing to itself (10.0.0.2) instead of the opnsense IP (10.0.0.1).

Code:
ip default-network 10.0.0.2/32
After making that change the switch can still ping out to the internet but the VLANs cannot.



EDIT: IT ****ING WORKS NOW. I am just stupid, and your advice pointed me in the right direction. I needed to change that default-network and also realized I never setup the route for the specific VLAN I am testing. I also had to make a change to the outbound NAT rule.
 
Last edited: