Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

LodeRunner

Active Member
Apr 27, 2019
553
235
43
Does anyone have experience with the ICX7450? How does it compare to the ICX6610? How loud is it? I need a switch that supports both 10Gb and 40Gb. I'm thinking between ICX6610, ICX7450 and VDX6740 (fiber models).
How many ports? A 7450 has 3 module slots, so a max of 3x 40Gb, 12x 10Gb or some combination thereof. Mine is in a rack with an Arista and several servers, so I couldn't tell you how loud it is; it doesn't seem to stick out versus anything else unless it gets into temp range 2 and sets the fans to 100%.
 
Last edited:
  • Like
Reactions: fohdeesha

dcplaya

New Member
Aug 1, 2016
1
0
1
36
Does anyone have iPXE booting working?

Currently, I have VLAN 2001 setup on the switch and it has `helper-address` set to my DHCP server (also my router, VyOS).
I can plug in a running device (Windows laptop) and I get a IP address from the correct DHCP. But when I boot a PXE device, it never gets an IP address and doesn't even seem to try!

This is not a LAGG port, just a standard port.

My config can be seen here

Any help would be greatly appreciated!
 

luks

New Member
Sep 23, 2021
8
2
3
Helsinki, Finland
How many ports? A 7450 has 3 module slots, so a max of 3x 40Gb, 12x 10Gb or some combination thereof. Mine is in a rack with an Arista and several servers, so I couldn't tell you how loud it is; it doesn't seem to stick out versus anything else unless it gets into temp raneg 2 and sets the fans to 100%.
Two 40Gb and 4x 10Gb would be enough.

ICX6610 seems to have higher power consumption on the datasheet. But it's a lot cheaper to buy
 

jasonwc

Member
Dec 31, 2018
49
18
8
The spec sheet for the ICX7450-48p shows 106W idle power consumption and 49dB acoustics. The acoustics are basically identical to an ICX6610-48p with dual B/C power supplies and dual fans (48.7dB per spec sheet). As for power consumption, the first post on this thread lists the ICX6610-48p at 110W idle. The spec sheet shows 165W for one PSU and 185w for two, but that doesn’t match real world idle usage.
 

Wolfcastle

Member
Jan 3, 2022
55
30
18
I just picked up a 7450-24p on ebay and have a 6610-24p on the way, I will be able to tell you soon exactly how they compare!
Does anyone have experience with the ICX7450? How does it compare to the ICX6610? How loud is it? I need a switch that supports both 10Gb and 40Gb. I'm thinking between ICX6610, ICX7450 and VDX6740 (fiber models).
 

bween

New Member
Dec 24, 2021
10
1
3
Hi, hoping someone can help with HomeKit and mDNS.
Setup:
- home assistant with homebridge to homekit, on servers vlan 10
- HomePod mini, on trusted vlan 20
- rpi running avahi-daemon, interface on vlan 10/20
- all routing done

Problem:
When my iPhone (connected to vlan 20) is on the home network, everything works fine. When my iPhone is not connected to the home network, all of the homebridge ONLY devices are not responding. Problem immediately goes away when the iPhone is connected to the home network.

Tried restarting devices, repairing things etc to no avail.

Is there some setting on the switch I need to enable for HomePod to recognize avahi as the mdns reflector? Maybe an ip helper address or something?
 

richtj99

Member
Jul 8, 2017
70
1
8
51
I have a Dahua POE IP camera on my 7250-48P. It stopped working & I am trying to troubleshoot.

When I plug it in, the POE does send power - is there a way to see what IP it has through a ssh session?

Not sure if it changed IP's or is working properly or at all. It seems to power up or at least the switch is sending power.

Any ideas on how to get more info from the switch?

Code:
SSH@7250#sh inline power
Power Capacity:         Total is 740000 mWatts. Current Free is 354200 mWatts.
Power Allocations:      Requests Honored 3209 times
 Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
 1/1/30 On      On          7300      30000  802.3at  Class 4     3  n/a
--------------------------------------------------------------------------
 Total                     76000     385800

SSH@7250(config)#sh int e 1/1/30
GigabitEthernet1/1/30 is up, line protocol is up
  Port up for 4 minute(s) 39 second(s)
  Hardware is GigabitEthernet, address is 609c.9f75.c9f4 (bia 609c.9f75.ca11)
  Configured speed auto, actual 100Mbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual MDIX
  EEE Feature Disabled
  Untagged member of L2 VLAN 68, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is config enabled, oper enabled, negotiation disabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  VLAN-Mapping is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  IPG MII 0 bits-time, IPG GMII 0 bits-time
  MTU 1500 bytes, encapsulation ethernet
  MMU Mode is Store-and-forward
  300 second input rate: 80 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 49016 bits/sec, 27 packets/sec, 0.04% utilization
  282536433 packets input, 396072306369 bytes, 0 no buffer
  Received 122868 broadcasts, 3887 multicasts, 282409678 unicasts
  12 input errors, 12 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  167684904 packets output, 14766722895 bytes, 0 underruns
  Transmitted 27103983 broadcasts, 5383352 multicasts, 135197569 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled
  Protected: No
  MAC Port Security: Disabled

UC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0           135042966                   0
         1                   0                   0
         2                   0                   0
         3                   0                   0
         4               11274                   0
         5                   7                   0
         6                   0                   0
         7              586058                   0


MC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0             4850420                 141
         1            27020055                   0
         2              170042                   0
         3                4087                   0

SSH@Office7250(config)#
 

LodeRunner

Active Member
Apr 27, 2019
553
235
43
Hi, hoping someone can help with HomeKit and mDNS.
Setup:
- home assistant with homebridge to homekit, on servers vlan 10
- HomePod mini, on trusted vlan 20
- rpi running avahi-daemon, interface on vlan 10/20
- all routing done

Problem:
When my iPhone (connected to vlan 20) is on the home network, everything works fine. When my iPhone is not connected to the home network, all of the homebridge ONLY devices are not responding. Problem immediately goes away when the iPhone is connected to the home network.

Tried restarting devices, repairing things etc to no avail.

Is there some setting on the switch I need to enable for HomePod to recognize avahi as the mdns reflector? Maybe an ip helper address or something?
Is the reflector enabled in your Avahi config?

Edit: to be honest, this is one of the reasons I flattened my network.
 

bween

New Member
Dec 24, 2021
10
1
3
Is the reflector enabled in your Avahi config?

Edit: to be honest, this is one of the reasons I flattened my network.
Thanks for the quick reply @LodeRunner.
Yes, reflector is enabled with “enable-reflector=yes” in /etc/avahi/avahi-daemon.conf. No other configuration edited from default though.

And yes, this issue is driving me nuts… seems like most guides/instructables out there suggest enabling reflector in opnsense or something but I’d like to get it working on the brocade if possible :(
 

LodeRunner

Active Member
Apr 27, 2019
553
235
43
All I can say is good luck. I got avahi mDNS reflector working on a VM with vNICs interfaces in both VLANs (no tagging or sub-/virtual interface in the VM). It worked about 75% of the time.

And you have properly tagged VLAN sub-interfaces configured on the rPi? And those interfaces are passing traffic? And there's no default firewall on the rPi?
 

bween

New Member
Dec 24, 2021
10
1
3
All I can say is good luck. I got avahi mDNS reflector working on a VM with vNICs interfaces in both VLANs (no tagging or sub-/virtual interface in the VM). It worked about 75% of the time.

And you have properly tagged VLAN sub-interfaces configured on the rPi? And those interfaces are passing traffic? And there's no default firewall on the rPi?
Yep, VLAN sub-interfaces seem to be working properly. Am able to ssh and connect to other devices through both interfaces on both VLANs. Default firewall is off. Did you ever figure out why it worked only sporadically?

Would you mind sharing more details on what you're doing now instead? Seems like bonjour and avahi might just be a headache not worth going deeper into.
 

jasonwc

Member
Dec 31, 2018
49
18
8
Is the reflector enabled in your Avahi config?

Edit: to be honest, this is one of the reasons I flattened my network.
Fortunately, Avahi is dead easy to setup on pfsense. You just select the interfaces you want the mdns packets to be reflected on and then check one box to enable repeating mdns packets across subnets. Once that's configured, I was able to control HomeKit devices across subnets without issues. I suspect the same can be done on the Brocade switch but I see no reason to fix what's already working.
 

jasonwc

Member
Dec 31, 2018
49
18
8
I have a Dahua POE IP camera on my 7250-48P. It stopped working & I am trying to troubleshoot.

When I plug it in, the POE does send power - is there a way to see what IP it has through a ssh session?

Not sure if it changed IP's or is working properly or at all. It seems to power up or at least the switch is sending power.

Any ideas on how to get more info from the switch?

Code:
SSH@7250#sh inline power
Power Capacity:         Total is 740000 mWatts. Current Free is 354200 mWatts.
Power Allocations:      Requests Honored 3209 times
Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
1/1/30 On      On          7300      30000  802.3at  Class 4     3  n/a
--------------------------------------------------------------------------
Total                     76000     385800

SSH@7250(config)#sh int e 1/1/30
GigabitEthernet1/1/30 is up, line protocol is up
  Port up for 4 minute(s) 39 second(s)
  Hardware is GigabitEthernet, address is 609c.9f75.c9f4 (bia 609c.9f75.ca11)
  Configured speed auto, actual 100Mbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual MDIX
  EEE Feature Disabled
  Untagged member of L2 VLAN 68, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is config enabled, oper enabled, negotiation disabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  VLAN-Mapping is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  IPG MII 0 bits-time, IPG GMII 0 bits-time
  MTU 1500 bytes, encapsulation ethernet
  MMU Mode is Store-and-forward
  300 second input rate: 80 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 49016 bits/sec, 27 packets/sec, 0.04% utilization
  282536433 packets input, 396072306369 bytes, 0 no buffer
  Received 122868 broadcasts, 3887 multicasts, 282409678 unicasts
  12 input errors, 12 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  167684904 packets output, 14766722895 bytes, 0 underruns
  Transmitted 27103983 broadcasts, 5383352 multicasts, 135197569 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled
  Protected: No
  MAC Port Security: Disabled

UC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0           135042966                   0
         1                   0                   0
         2                   0                   0
         3                   0                   0
         4               11274                   0
         5                   7                   0
         6                   0                   0
         7              586058                   0


MC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0             4850420                 141
         1            27020055                   0
         2              170042                   0
         3                4087                   0

SSH@Office7250(config)#
I have a bunch of Dahua cameras in a dedicated VLAN for the security camera network. I found them very easy to setup and haven't had any issues with them - but the very first thing I did was setup a static IP address. Since the port shows as up on the switch, I would check your DHCP server logs to see what IP address it assigned. The most likely scenario is that the router assigned a new IP. To find the MAC address of the attached camera, try:

show mac-address e 1/1/30

Another option would be to check the ARP table on the router. You could also do a scan for all addresses on the subnet depending on how many cameras you have on that VLAN. Once you find the IP, login and manually assign an IP.

I also find it odd that the inline power statistics shows the device is using 802.3at. Every Dahua camera I've seen lists 802.3af, including the PTZ cameras. My Dahua cameras all show 802.3af in inline power details.

1/1/3 On On 2600 15400 802.3af n/a 3 n/a
1/1/29 On On 3200 15400 802.3af n/a 3 n/a
1/1/30 On On 2200 15400 802.3af n/a 3 n/a
1/1/31 On On 5900 15400 802.3af n/a 3 n/a
 
Last edited:

bween

New Member
Dec 24, 2021
10
1
3
Fortunately, Avahi is dead easy to setup on pfsense. You just select the interfaces you want the mdns packets to be reflected on and then check one box to enable repeating mdns packets across subnets. Once that's configured, I was able to control HomeKit devices across subnets without issues. I suspect the same can be done on the Brocade switch but I see no reason to fix what's already working.
Are you using pfsense for all routing with brocade in L2? I tried enabling IGMP snooping actively/passively on brocade to see if that'd help forward the mDNS along, but that didn't work, and I don't really understand enough about the nitty gritty to get any deeper into it. Maybe I'll just need to move home assistant over to the trusted user vlan so it can bridge stuff to homepod properly.
 

jasonwc

Member
Dec 31, 2018
49
18
8
Are you using pfsense for all routing with brocade in L2? I tried enabling IGMP snooping actively/passively on brocade to see if that'd help forward the mDNS along, but that didn't work, and I don't really understand enough about the nitty gritty to get any deeper into it. Maybe I'll just need to move home assistant over to the trusted user vlan so it can bridge stuff to homepod properly.
PFSense handles all routing with the exception of traffic between two VLANs with 10G clients. The IoT VLAN (VLAN 2) that contains my Phillips Hue hub is routed by PFSense. It's accessed primarily by my trusted WPA3-Enterprise WLAN network (VLAN 11) and limited WLAN network (VLAN 9). Even with 10G traffic, PFSense had no issues routing traffic at wirespeed using jumbo packets at 10G (~10% CPU utilization). Given that it's easier to manage all the firewall rules in one place in PFSense, I'm planning to move all my 10G and 40G clients to one VLAN for simplicity, as shown below.

PFSense Hardware:
Xeon E5-1220v2 3.1Ghz
8GB DDR3
Intel X520-DA2 with two SFP+ ports
FS QSFP+ to 4x SFP+ breakout cable connects from the X520-DA2 to my ICX6610-48p for WAN and LAN
LAN contains untagged traffic for the trusted LAN and 14 tagged VLANs
 

Attachments

Last edited:
  • Like
Reactions: carbon60

bween

New Member
Dec 24, 2021
10
1
3
PFSense handles all routing with the exception of traffic between two VLANs with 10G clients. The IoT VLAN (VLAN 2) that contains my Phillips Hue hub is routed by PFSense. It's accessed primarily by my trusted WPA3-Enterprise WLAN network (VLAN 11) and limited WLAN network (VLAN 9). Even with 10G traffic, PFSense had no issues routing traffic at wirespeed using jumbo packets at 10G (~10% CPU utilization). Given that it's easier to manage all the firewall rules in one place in PFSense, I'm planning to move all my 10G and 40G clients to one VLAN for simplicity, as shown below.

PFSense Hardware:
Xeon E5-1220v2 3.1Ghz
8GB DDR3
Intel X520-DA2 with two SFP+ ports
FS QSFP+ to 4x SFP+ breakout cable connects from the X520-DA2 to my ICX6610-48p for WAN and LAN
LAN contains untagged traffic for the trusted LAN and 14 tagged VLANs
Goodness gracious haha. I appreciate the thorough write up/network diagram. Interesting that you use the brocade to interface with WAN. Any benefit to doing that rather than letting opn/pfsense do that job?

Still hoping I'm able to solve the original problem with homekit and mdns - feels like the solution shouldn't be this hard but alas I feel stuck. And mostly I'd like to avoid busting up the network again for the family. Setting everything up with opnsense may have to be a last resort kind of deal :)
 

jasonwc

Member
Dec 31, 2018
49
18
8
Goodness gracious haha. I appreciate the thorough write up/network diagram. Interesting that you use the brocade to interface with WAN. Any benefit to doing that rather than letting opn/pfsense do that job?

Still hoping I'm able to solve the original problem with homekit and mdns - feels like the solution shouldn't be this hard but alas I feel stuck. And mostly I'd like to avoid busting up the network again for the family. Setting everything up with opnsense may have to be a last resort kind of deal :)
Future-proofing. AT&T FTTH is already offering 2Gb symmetrical and 5Gb symmetrical connections in parts of the country and internal sources at Verizon indicate multi-gig services are expected to be offered for FiOS this year. I signed up for gigabit FiOS the day it was announced in 2017 and look forward to faster speeds soon. Verizon has been working with Calix to provide their NG-PON2 ONTs (40G/40g PON versus 2.5/1.2G GPON) and Calix has provided specifications on an NG-PON2 ONT that appears to be designed for Verizon. It has a 10GBase-T port. Because the Intel X520-DA2 is much more picky about transceivers, I would rather connect the ONT to the ICX6610 using a SFP+ 10GBase-T transceiver. From the first post on this thread, I already know the ICX6610 will work with a Mikrotik S+RJ10 SFP+ 10gbase-T copper module. The Intel card will accept any DAC, so I used a breakout QSFP+ DAC to connect the ICX6610-48p port (1/2/2 - 1/2/5) to both SFP+ ports on the X520-DA2. That leaves the SFP+ ports at the front for my 10G-LR connections and I have two more SFP+ DACs I can use if needed in the future.

I could have gone with the Intel X540-T2 but it's stupidly power hungry (something like 17.5W) versus a maximum of 6W on the X520-DA2 (less with DACs which use 0.1W each) or 3.3W on the Mellanox ConnectX-3 single SFP+ cards. In addition, I just didn't want to deal with 10GBase-T for ports that didn't need it. DACs are lower latency and use way less power. SMF is great for everything long-range since it'll support 10/25/40/50/100/200/400G speeds over duplex cables, and there are already affordable 25G and 100G optics. 10G and 40G optics are actually cheaper than MMF for long runs, especially for 40G, where you would need to run MPO cable, which gets very expensive when you're running 100+ ft.
 
Last edited:

bween

New Member
Dec 24, 2021
10
1
3
Future-proofing. AT&T FTTH is already offering 2Gb symmetrical and 5Gb symmetrical connections in parts of the country and internal sources at Verizon indicate multi-gig services are expected to be offered for FiOS next year. I signed up for gigabit FiOS the day it was announced in 2017 and look forward to faster speeds soon. Verizon has been working with Calix to provide their NG-PON2 ONTs (40G/40g PON versus 2.5/1.2G GPON) and Calix has provided specifications on an NG-PON2 ONT that appears to be designed for Verizon. It has a 10GBase-T port. Because the Intel X520-DA2 is much more picky about transceivers, I would rather connect the ONT to the ICX6610 using a SFP+ 10GBase-T transceiver. From the first post on this thread, I already know the ICX6610 will work with a Mikrotik S+RJ10 SFP+ 10gbase-T copper module. The Intel card will accept any DAC, so I used a breakout QSFP+ DAC to connect the ICX6610-48p port (1/2/2 - 1/2/5) to both SFP+ ports on the X520-DA2. That leaves the SFP+ ports at the front for my 10G-LR connections and I have two more SFP+ DACs I can use if needed in the future.

I could have gone with the Intel X540-T2 but it's stupidly power hungry (something like 17.5W) versus a maximum of 6W on the X520-DA2 (less with DACs which use 0.1W each) or 3.3W on the Mellanox ConnectX-3 single SFP+ cards. In addition, I just didn't want to deal with 10GBase-T for ports that didn't need it. DACs are lower latency and use way less power. SMF is great for everything long-range since it'll support 10/25/40/50/100/200/400G speeds over duplex cables, and there are already affordable 25G and 100G optics. 10G and 40G optics are actually cheaper than MMF for long runs, especially for 40G, where you would need to run MPO cable, which gets very expensive when you're running 100+ ft.
Very neat! ATT only recently brought 1g symmetrical to my neck of the woods. Still a while yet till we get anything faster I think.
 

Vesalius

Active Member
Nov 25, 2019
261
202
43
Very neat! ATT only recently brought 1g symmetrical to my neck of the woods. Still a while yet till we get anything faster I think.
And ATT makes it near impossible to connect their ONT directly to a switch SFP+ transceiver, especially for greater than 1Gb service as the ont is built into the modem.