Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
R

rocketpanda40

Member
Dec 12, 2019
51
33
18
craig5571 said:
thank you so much , will this allow intervlan routing as well for example three vlans 10 20 and 30
all on different subnets 192.168.10.0 , 20.0, 30.0 etc..

will a device on 20.0 be able to ping 10.0 and 30.0 and vice versa

i had thought i read something about having to have a ve interface

BROCADE CAMPUS FEATURE EXPLAINER SERIES - VLAN ROUTING WITH ROUTER-INTERFACE CONFIGURATION


I should know this all by heart, but I been doing Wi-Fi so long.. alot of simple wired stuff has slipped my mind..

thanks again
Click to expand...
If you want the switch to do inter-vlan routing, you'll need the virtual ethernet interfaces, yeah.

That's as simple as

Code: 
vlan 1
router-interface ve 1
int ve 1
ip add 10.1.1.1/24

vlan 2
router-interface ve 2
int ve 2
ip add 10.2.2.1/24
Now the switch will route between 10.1.1.0/24 and 10.2.2.0/24.

Here's one of my complete, working configs of an ICX7250-48P acting as my home's core switch. There's a mix of vlans being routed by the switch and others in L2 only, being routed and firewalled by vyos upstream. Also present is OSPF, LACP LAGs, etc. This is before I added a few simple ACLs though, but should help in translating the Cisco IOS knowledge to Fastiron.

It will probably help understanding to know that:

  • I've connected my wan straight to the switch on a vlan, and have my firewall's wan interface also on that vlan
  • Port with name "B2" goes to another switch I have connecting my workstation (with its own lab), an access point, and a printer, so that switch is also running L3 firmware
  • "wlc" is my wireless controller
  • "vyos" is my firewall
  • OSPF exchanges routes between all of the aforementioned
  • The LAG interfaces are all 2x1Gb links to my proxmox servers. I'm not currently using the 10Gb interfaces because after massively downscaling my lab, I now have like 30 10Gb DACs and 0 10Gb PCIe cards (I had a whole bunch of dual 10Gb OCP cards that I sold with my old servers)
 
  • Like
Reactions: craig5571
A

acpatel

New Member
Sep 8, 2020
5
0
1
rocketpanda40 said:
The UFI images automatically flash the PoE firmware - watch the console output, you'll see it. But when you download the firmware, you'll get the separate poe image as well if you want to manually flash it for assurance.
Click to expand...
Thoughts on the seg fault?
 
C

craig5571

Member
May 31, 2020
60
6
8
rocketpanda40 said:
If you want the switch to do inter-vlan routing, you'll need the virtual ethernet interfaces, yeah.

That's as simple as

Code: 
vlan 1
router-interface ve 1
int ve 1
ip add 10.1.1.1/24

vlan 2
router-interface ve 2
int ve 2
ip add 10.2.2.1/24
Now the switch will route between 10.1.1.0/24 and 10.2.2.0/24.

Here's one of my complete, working configs of an ICX7250-48P acting as my home's core switch. There's a mix of vlans being routed by the switch and others in L2 only, being routed and firewalled by vyos upstream. Also present is OSPF, LACP LAGs, etc. This is before I added a few simple ACLs though, but should help in translating the Cisco IOS knowledge to Fastiron.

It will probably help understanding to know that:

  • I've connected my wan straight to the switch on a vlan, and have my firewall's wan interface also on that vlan
  • Port with name "B2" goes to another switch I have connecting my workstation (with its own lab), an access point, and a printer, so that switch is also running L3 firmware
  • "wlc" is my wireless controller
  • "vyos" is my firewall
  • OSPF exchanges routes between all of the aforementioned
  • The LAG interfaces are all 2x1Gb links to my proxmox servers. I'm not currently using the 10Gb interfaces because after massively downscaling my lab, I now have like 30 10Gb DACs and 0 10Gb PCIe cards (I had a whole bunch of dual 10Gb OCP cards that I sold with my old servers)
Click to expand...
major thank you!! , have you modified the fans at all? I can totally live with the 7250P stock , but if I could make it quieter and keep it cool ( the most important thing ) that would be nice.. ( the switch is on the shelf about six feet to my left at approximately ear height.. )

I was thinking of replacing the two stock fans
https://forums.servethehome.com/proxy.php?image=https%3A%2F%2Fi.imgur.com%2Fr8Kd6dH.png&hash=451a90f836a49a4837fa8bbc33a7eb36

with three of these

  • Comair Rotron "Gryphon" GDA4028-12BB - discontinued, alas
    If you can find these? BUY THESE. 40x40x28mm, but 11CFM, 0.34in H2O, 8800RPM, but just 31.4dBA! I am still mad they discontinued them. No, I am not selling any of my spares.
I got 3 of them.. but they came with bare wires.. so I ordered ends to put on them..

pic of the fan

GDA4028-12BB Comair Rotron | Fans, Thermal Management | DigiKey

Order today, ships today. GDA4028-12BB – Fan Tubeaxial 12VDC Square - 40mm L x 40mm H Ball 11.0 CFM (0.308m³/min) 3 Wire Leads from Comair Rotron. Pricing and Availability on millions of electronic components from Digi-Key Electronics.
www.digikey.com www.digikey.com
3 Wires - (Red-positive, Black-negative, White-Tachometer)

I'm looking for the wire diagram for the original fans to know which wire is which wire.. the original fans three wires green yellow and black ( from left to right) but I dont know what green= or yellow= or black= , I assume positive, negative and tachometer.

http://imgur.com/a/4z1idZp

ICX7250-24P Internals is the one I have..

in post #4275 fan options are discussed very in depth
" If you want to quiet down these units, the BIGGEST change you can make is to remove the stamped grill with a Dremel. It's all sharp edges, and significantly blocks the hub as well. Turbulence from things like grills are what generate serious noise. If you still need protection, tack on a wire grill on the OUTSIDE of the chassis "

the Stamped Grill is the circular holes where the air comes out on the back ? ( at least thats how I think...)


and if all that made the system quieter.. but not cool enough...
I was thinking of adding a fractal design 140 mm fan ( i have an extra one) on the top of the case blowing down in the case ,( i test this out by getting a sheet of plastic from lowes cutting that to verify the idea works before , modding the case ) the other 3 fans would blow the air out..

I haven't done any of this fan work yet.. just brain storming.. I sometimes get things wrong.. but alot of times I get it right.
 
Last edited:
T

tubs-ffm

Active Member
Sep 1, 2013
210
70
28
craig5571 said:
I'm looking for the wire diagram for the original fans to know which wire is which wire.. the original fans three wires green yellow and black ( from left to right) but I dont know what green= or yellow= or black= , I assume positive, negative and tachometer.
Click to expand...
I replaced the original Foxconn PIA040H12P in my 7250-24P yesterday and was facing the same issue. I used google and found some standard wiring definitions that fit:

black --> -
yellow --> +
green --> frequency

Only the position on the connector of the Ruckus was not matching to the standard definition. I wired the new fan to match to the cable position of the original fan.

Edit: typos corrected
 
Last edited:
  • Like
Reactions: craig5571
I

ipmifreely

New Member
Jan 16, 2021
5
0
1
ipmifreely said:
inter vlan routing WITHOUT doing it on the switch (icx6610)

hi all,

I've just upgraded from a HP procurve with layer 2 vlan support and my intervlan routing is not working - it was working fine with the HP. I'm seeing a lot of folks in this thread with similar problems but (i think) they are mostly trying to solve the problem IN switch and i want to keep my nice opnsense GUI....

my setup: I have a hand full of vlans and Opnsense running as a VM on proxmox. Proxmox has and LACP/LAG bonded connection to the switch. so

the setup of the switch is

vlan 3000 - wan
vlan 1 - management
vlan 2 - lan
vlan 4 - wifi
vlan 10 - defualt_vlan (i reassigned it)

for the wan iput port i set it as dual mode becuase my ISP is GPON and sets the services on speific vlans (yes i probably don't need this - but i don't want that port part of the default vlan and don't know how else to do it)
for the LACP ports for proxmox i set vlan y as dual mode (because i couldn't get untagged to work with LACP) and the rest of the vlans to tagged.
for the rest of the 1st 22 ports i set vlan 1 as untagged
for ports 22&24 i set vlan 4 (wifi) as untaggged
for ports 25-48 i set vlan 2 as untagged.

this is more or less and exact replication of what i was doing with my procurve (the new switch just has more ports so i've spread things about a bit)

The setup of Proxmox is:

i create the bond
i setup all the vlans i need.
i create bridge interfaces for all the vlans
i share the bridge interfaces for all the vlans with my VM and to them they look like separate adapters...

The setup inside the opnsense VM is:
is sees all the bridge interfaces from poxmox as network adapters and applys firewall and routing between them.

yes i know that the icx is layer 3 - I got the ICX6610 for more ports, 10 & 40 Gig, and POE and not so much for Layer 3 at it is overwhelming for me.

am i being thick ? i searched the forum already but didn't find anything that seemed to pertain to my sistuation (routing from a router that is a VM). if my problem is easily googilable please just point me in the right direction and accept my appologies in advance :)
Click to expand...
i thought i might have messed up earlier in my config by following fodesha's acl guide but show running-config dosn't list any ACLs so thats not it...
 
M

-MoNsTeRRR

New Member
May 24, 2020
5
1
3
Hello,

I've tried today to configure IPv6 ACL on my ICX 6650 and i've faced this issue when i've applyed an ACL :
Code: 
Insufficient hardware resources to apply the V6 ACL. Please remove already applied ACL(s) and/or Security features and try again.
I've 6 ACL for IPv4 and 6 ACL for IPv6. I've applyed all IPv4 ACL and 5 IPv6 ACL and can't apply the last.

I've tried to increase system-max values but it seems ip-filter-sys & ip-filter-port doesn't resolve the issue with max values.

I've deleted all the rules in the ACL, i've added the ACL to the ve interface and tried to reapply the rules but I faced this error :
Code: 
ERROR: Insufficient hardware resource for binding the ACL to interface v10.
Here is the IPv6 & IPv4 ACL applyed to ve10
Code: 
ipv6 access-list DMZ_IN
remark "ALLOW ANY to ANY ESTABLISHED"
permit tcp any any established
remark "ALLOW ANY to ANY ICMP ECHO-REPLY"
permit icmp any any echo-reply
remark "ALLOW DHCP1-REPLY to ANY"
permit udp host 2a0c:b641:2c0:110::23 any eq 68
remark "ALLOW DHCP2-REPLY to ANY"
permit udp host 2a0c:b641:2c0:110::24 any eq 68
remark "ALLOW DNS1/UDP to ANY"
permit udp host 2a0c:b641:2c0:110::21 eq 53 any
remark "ALLOW DNS2/UDP to ANY"
permit udp host 2a0c:b641:2c0:110::22 eq 53 any
remark "ALLOW VM-MONITORING to VLAN-MANAGEMENBT SNMP"
permit udp host 2a0c:b641:2c0:110::101 2a0c:b641:2c0:140::/64 eq 161
remark "ALLOW VM-MONITORING to LAB NODE-EXPORTER"
permit tcp host 2a0c:b641:2c0:110::101 2a0c:b641:2c0:120::/64 eq 9100
remark "ALLOW VM-MONITORING to LAB VMWARE-EXPORTER"
permit tcp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:140::10 eq 9272
remark "ALLOW VM-MONITORING to UNIFI-POLLER"
permit tcp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:120::50 eq 8443
remark "ALLOW VM-MONITORING to NAS-SNMP"
permit udp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:120::42 eq 161
remark "ALLOW VM-MONITORING to HS110-RACK1"
permit tcp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:150::101 eq 9233
remark "ALLOW VM-MONITORING to HS110-RACK2"
permit tcp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:150::102 eq 9233
remark "ALLOW VM-MONITORING to HS110-CHAMBRE1"
permit tcp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:150::103 eq 9233
remark "ALLOW VM-MONITORING to WIREGUARD-EXPORTER"
permit tcp host 2a0c:b641:2c0:110::101 host 2a0c:b641:2c0:170::29 eq 9998
remark "DENY ROUTING"
deny ipv6 any 2a0c:b641:2c0:105::/64
remark "DENY LOOPBACK"
deny ipv6 any 2a0c:b641:2c0:106::/64
remark "DENY VLAN-LAB"
deny ipv6 any 2a0c:b641:2c0:120::/64
remark "DENY VLAN-ADMIN"
deny ipv6 any 2a0c:b641:2c0:130::/64
remark "DENY VLAN-MANAGEMENT"
deny ipv6 any 2a0c:b641:2c0:140::/64
remark "DENY VLAN-HOME"
deny ipv6 any 2a0c:b641:2c0:150::/64
remark "DENY VLAN-GUEST"
deny ipv6 any 2a0c:b641:2c0:160::/64
remark "ALLOW ANY to ANY"
remark "DENY VLAN-VPN"
deny ipv6 any 2a0c:b641:2c0:170::/64
permit ipv6 any any


ip access-list extended DMZ_IN
remark "ALLOW ANY to ANY ESTABLISHED"
permit tcp any any established
remark "ALLOW ANY to ANY ICMP ECHO-REPLY"
permit icmp any any echo-reply
remark "ALLOW DHCP1-REPLY to ANY"
permit udp host 192.168.10.23 any eq 68
remark "ALLOW DHCP2-REPLY to ANY"
permit udp host 192.168.10.24 any eq 68
remark "ALLOW DNS1/UDP to ANY"
permit udp host 192.168.10.21 eq 53 any
remark "ALLOW DNS2/UDP to ANY"
permit udp host 192.168.10.22 eq 53 any
remark "ALLOW VM-MONITORING to VLAN-MANAGEMENBT SNMP"
permit udp host 192.168.10.101 192.168.40.0 0.0.0.255 eq 161
remark "ALLOW VM-MONITORING to LAB NODE-EXPORTER"
permit tcp host 192.168.10.101 192.168.20.0 0.0.0.255 eq 9100
remark "ALLOW VM-MONITORING to LAB VMWARE-EXPORTER"
permit tcp host 192.168.10.101 host 192.168.40.10 eq 9272
remark "ALLOW VM-MONITORING to UNIFI-POLLER"
permit tcp host 192.168.10.101 host 192.168.20.50 eq 8443
remark "ALLOW VM-MONITORING to NAS-SNMP"
permit udp host 192.168.10.101 host 192.168.20.42 eq 161
remark "ALLOW VM-MONITORING to HS110-RACK1"
permit tcp host 192.168.10.101 host 192.168.50.101 eq 9233
remark "ALLOW VM-MONITORING to HS110-RACK2"
permit tcp host 192.168.10.101 host 192.168.50.102 eq 9233
remark "ALLOW VM-MONITORING to HS110-CHAMBRE1"
permit tcp host 192.168.10.101 host 192.168.50.103 eq 9233
remark "ALLOW VM-MONITORING to WIREGUARD-EXPORTER"
permit tcp host 192.168.10.101 host 192.168.70.29 eq 9998
remark "DENY ROUTING"
deny ip any 192.168.5.0 0.0.0.255
remark "DENY LOOPBACK"
deny ip any 192.168.6.0 0.0.0.255
remark "DENY VLAN-LAB"
deny ip any 192.168.20.0 0.0.0.255
remark "DENY VLAN-ADMIN"
deny ip any 192.168.30.0 0.0.0.255
remark "DENY VLAN-MANAGEMENT"
deny ip any 192.168.40.0 0.0.0.255
remark "DENY VLAN-HOME"
deny ip any 192.168.50.0 0.0.0.255
remark "DENY VLAN-GUEST"
deny ip any 192.168.60.0 0.0.0.255
remark "ALLOW ANY to ANY"
remark "DENY VLAN-VPN"
deny ip any 192.168.70.0 0.0.0.255
permit ip any any

interface ve 10
 ip access-group DMZ_IN in
 ip address 192.168.10.2 255.255.255.0
 ip helper-address 1 192.168.10.23
 ip helper-address 2 192.168.10.24
 ipv6 address 2a0c:b641:2c0:110::2/64
 ipv6 enable
 ipv6 traffic-filter DMZ_IN in
 ipv6 dhcp-relay destination 2a0c:b641:2c0:110::23
 ipv6 dhcp-relay destination 2a0c:b641:2c0:110::24
Is it a limitation or am I doing wrong ?

Thanks :)
 
S

SIlviu

Member
May 27, 2016
84
8
8
Got a 6450 48 port on ebay with POE dead, I want to use it without POE, I removed the POE board, no more errors in CLI but the fans are 100% all the time. Is there a way to disable the POE on the device or to slow down the fans ?
 
O

Originalus

Member
Dec 22, 2020
47
21
8
i am interested in ICX7150-C12P, but have two main questions.
1. can it be used as router? looking at spec sheets and posts looks like can. maybe someone is running it like router.
2. can sfp+ and rj45 uplink ports used as simple ports for wiring lan devices?

one extra:
i have gpon fiber at home. Would it be possible to use gpon sfp transceiver on sfp ports?
 
W

Wronglebowski

New Member
Jun 18, 2018
5
0
1
There seems to be multiple variants of the "Brocade Official SFP+". Is there any particular model I need to get the added benefits?

www.ebay.com

Brocade 8gb FC 850nm SW SFP GBIC 57-1000117-01 for sale online | eBay

Find many great new & used options and get the best deals for Brocade 8gb FC 850nm SW SFP GBIC 57-1000117-01 at the best online prices at eBay! Free shipping for many products!
www.ebay.com

www.ebay.com

Brocade 10G-SFPP-SR 57-0000075-01 10GB 10GBASE-SR SFP module | eBay

Find many great new & used options and get the best deals for Brocade 10G-SFPP-SR 57-0000075-01 10GB 10GBASE-SR SFP module at the best online prices at eBay! Free shipping for many products!
www.ebay.com
 
R

rocketpanda40

Member
Dec 12, 2019
51
33
18
Originalus said:
i am interested in ICX7150-C12P, but have two main questions.
1. can it be used as router? looking at spec sheets and posts looks like can. maybe someone is running it like router.
2. can sfp+ and rj45 uplink ports used as simple ports for wiring lan devices?

one extra:
i have gpon fiber at home. Would it be possible to use gpon sfp transceiver on sfp ports?
Click to expand...
1. Yes, in the sense that you can run L3 firmware. It won't do NAT and extended ACLs are a pain compared to 'normal' firewall rules so it won't replace your home router/firewall if that's what you mean.
2. Yup
 
Last edited:
R

rocketpanda40

Member
Dec 12, 2019
51
33
18
Wronglebowski said:
There seems to be multiple variants of the "Brocade Official SFP+". Is there any particular model I need to get the added benefits?

www.ebay.com

Brocade 8gb FC 850nm SW SFP GBIC 57-1000117-01 for sale online | eBay

Find many great new & used options and get the best deals for Brocade 8gb FC 850nm SW SFP GBIC 57-1000117-01 at the best online prices at eBay! Free shipping for many products!
www.ebay.com

www.ebay.com

Brocade 10G-SFPP-SR 57-0000075-01 10GB 10GBASE-SR SFP module | eBay

Find many great new & used options and get the best deals for Brocade 10G-SFPP-SR 57-0000075-01 10GB 10GBASE-SR SFP module at the best online prices at eBay! Free shipping for many products!
www.ebay.com
Click to expand...
The first link are 8Gb Fiber channel transceivers. You want the second link.
 
  • Like
Reactions: tommybackeast and Wronglebowski
P

plexisaurus

New Member
Jan 14, 2021
6
2
3
I recently purchased a new icx 7250-48 off ebay. I'm experimenting with fan modding because stock ones are too loud for me. Can anyone hazard a educated guess what a safe 24/7 temp is? it was 65-70c at stock idle, now it is around 77c after 100minutes idle and not quite reached steady state. I know these have a limit of 100-105, but my experience with other silicon/laptops/pcs is that staying just below peak limit still isn't great long term.

FYI I've swapped the two stock fans with two Sunon MF40201vx-1000u-g99 MagLev ones. I'm considering adding a 3rd, adding a slim fan on top of asic, or go full nuclear and cut out holes for mounting dual top-down 140-180mm fans
 
K

kapone

Well-Known Member
May 23, 2015
1,356
802
113
I think this has been repeated enough in this thread (and others) enough times.

THERE IS NO GOOD WAY TO SILENCE THESE BEASTS.

You're talking serious enterprise gear than can push >0.5tbps. That's terabit per second.

Silence was never even a consideration.
 
P

plexisaurus

New Member
Jan 14, 2021
6
2
3
kapone said:
I think this has been repeated enough in this thread (and others) enough times.

THERE IS NO GOOD WAY TO SILENCE THESE BEASTS.

You're talking serious enterprise gear than can push >0.5tbps. That's terabit per second.

Silence was never even a consideration.
Click to expand...
THAT IS QUITTER TALK. Anything can be cooled quietly, just a matter of engineering and design trade offs. This isn't rocket science. I'm sure slapping two 180mm fans on top would do it easily with lower than stock temps, but it is more work, and would make it take up more rack space. Someone else did this with a ICX 6610 which has a higher TDP.
 
I

ipmifreely

New Member
Jan 16, 2021
5
0
1
ipmifreely said:
i thought i might have messed up earlier in my config by following fodesha's acl guide but show running-config dosn't list any ACLs so thats not it...
Click to expand...
doh!

so i've discoved a couple of things about fastiron:
1) you have to 'depoly' a lacp/lag bond before it activates
2) it seems less tolerant of janky setups than my old hp2810/J9021A

I discovered this my moving my router/firewall vm over to my 2nd host - all of a sudden my wifi (which is isolated on its own vlan) started working! it seems like something in the networking of one of my 2 hosts networking is borked. i'm having a look but i might also just give up and get 10gb nics for it and start again (as i'd been meaning to do this now that i have 10g ports to play with anyhow). i'm going to cross post to the proxmox forum in case those folks have any idea whats wrong - the two networking setups on the machine are exactly the same except for the systemd defined interface names (and it was working with the HP switch)

i'm having second thoughts about routing EVERY thing though my firewall/router VM - i might try my hand a doing from routing between my management vlan and my lan vlan as that is the only place where the performance would matter.

after I've got my misbehaving node figured out - I'll have a shot at pulling the sfp module our of my ONT as i have a similar ISP to this post (without the PPPoE BS) https://forums.servethehome.com/index.php?threads/brocade-icx-6610-with-bell-canada-fibe-ftth.20998/

so to answer my own question: if you don't define any router interfaces or ACLs an ICX series switch behaves like a layer 2 managed switch. (at least it seams to)
 
E

epicurean

Active Member
Sep 29, 2014
788
81
28
  • Like
Reactions: itronin
K

kapone

Well-Known Member
May 23, 2015
1,356
802
113
plexisaurus said:
THAT IS QUITTER TALK. Anything can be cooled quietly, just a matter of engineering and design trade offs. This isn't rocket science. I'm sure slapping two 180mm fans on top would do it easily with lower than stock temps, but it is more work, and would make it take up more rack space. Someone else did this with a ICX 6610 which has a higher TDP.
Click to expand...
Yup, I was one of the first ones to mod a 6610 with 3x 120mm fans on top. :)

Like I said no good way...
 
You must log in or register to reply here.
Top Bottom