Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[MrSliff]

I'm searching for Service Manuals and Schematics for the ICX7250 POE Version Switch.

Does anybody know where i can find these? Ive seen a defective switch where one MOSFET in the intermediate circuit of the power supply exploded, also one Capacitor is damaged by the explosion. I may get it if i can get Service Documents. These Documents would make the repair process much easier, since i could look for the right replacement part. I dont know if the text on the second MOSFET is still readable (would replace both then).

 

[kpfleming]

I'm searching for Service Manuals and Schematics for the ICX7250 POE Version Switch.

Does anybody know where i can find these? Ive seen a defective switch where one MOSFET in the intermediate circuit of the power supply exploded, also one Capacitor is damaged by the explosion. I may get it if i can get Service Documents. These Documents would make the repair process much easier, since i could look for the right replacement part. I dont know if the text on the second MOSFET is still readable (would replace both then).

The place to ask would be the Ruckus Wireless forums.

 

[MrSliff]

The place to ask would be the Ruckus Wireless forums.

Thanks, i will check if i can get them there.

 

[withoutink]

Brand new here, have yet to read through the 329 pages, but I did ping pong around some search results in this thread. I am considering purchasing a Ruckus ICX7150 - C12P - 2X1G from eBay, but wondering if that is advisable. I saw a couple mentions of PSU's dying, and since these are not warrantied past their original purchaser. I pinged the seller, and they said its in like new condition, and is working perfectly.

I don't mind purchasing, if this won't be a headache down the road.

If the PSU is the only issue what does it cost to replace the PSU?

Thanks in advance

 

[kpfleming]

Brand new here, have yet to read through the 329 pages, but I did ping pong around some search results in this thread. I am considering purchasing a Ruckus ICX7150 - C12P - 2X1G from eBay, but wondering if that is advisable. I saw a couple mentions of PSU's dying, and since these are not warrantied past their original purchaser. I pinged the seller, and they said its in like new condition, and is working perfectly.

I don't mind purchasing, if this won't be a headache down the road.

If the PSU is the only issue what does it cost to replace the PSU?

Thanks in advance

I bought four, one died about 48 hours after I received it and plugged it in (PSU failure), seller replaced it and all four have been fine for months since. Given their capabilities and the relatively low cost on eBay, buy at least one more than you actually need and test them all well

 

[RobstarUSA]

Hi folks. Long time lurker, first time poster. I have a recently acquired Brocade 6610 that seems to get in a pissing match with my server every morning before it just stops talking to it like a jilted lover. Wondering if anyone else has had similar issues.

Server is running latest edition of Proxmox.
NIC is a Mellanox CX354-FCCT, latest firmware. Port being used is set to ethernet mode. Used from ebay, but shows no problems elsewhere.
Cable is a FS.com QSFP+ 40gbps passive DAC. Brand new from vendor.
Switch is a 6610-48p, updated and fully licensed thanks to a rockstar member here we all know and appreciate. Used from a local sale.

At some point overnight, the 40gb connection will start to flap, up and down within a second, every second until link error disable threshold is reached and the port goes into err-disable for 5 seconds. It'll repeat the cycle until the port is eventually released from err-disable, but the link doesn't come back up. Then I wake up and go replug the DAC and it works fine again all day. Looking up the statistics on the switch shows several hundred InErrors accumulate, but that's all. The server shows no signs anything is wrong (other than losing connection). If it's worth noting, during the day the interface will accumulate a few more InErrors (less than 10) but with no effect on performance or link. Heavy traffic doesn't seem to be the issue, as it successfully performed a VM backup over the connection about an hour before the incident.

I've tried switching NICs and DACs for identical models and configuration. Tried changing to the other QSFP port (yes the 40gb port not the breakouts). Flow control is enabled on the switch and NIC (ethtool says the interface supports pause frame use Symmetric Receive-only, but ethtool -a lists TX and RX = on). Link error disable is set on the interface with a 5 second recovery time. Errdisable recovery is set for all causes and 10 seconds recovery. Network is flat, no VLANs or special configurations apart from the error handling mentioned.

Unfortunately I don't have another switch to test with, but directly linking two servers together on the same port/DAC results in a stable connection with no hiccups.

Has anyone had a similar issue before, or am I chasing ghosts when they're in the house next door? Any troubleshooting tips would be appreciated as well.

Have you tried disabling flow control? I've never seen it work well in a modern network.
Edit: I don't think this will fix your L1 issues however. In any case I have propxmox ve on a 10Gbit/s link & it's rock solid, so I don't think it's anythign with proxmox but perhaps an transceiver/nic/switch compatibility issue. What device does the nic show up as in proxmox? Which kernel are you on ?

 

[luks]

Does anyone have experience with the ICX7450? How does it compare to the ICX6610? How loud is it? I need a switch that supports both 10Gb and 40Gb. I'm thinking between ICX6610, ICX7450 and VDX6740 (fiber models).

 

[LodeRunner]

Does anyone have experience with the ICX7450? How does it compare to the ICX6610? How loud is it? I need a switch that supports both 10Gb and 40Gb. I'm thinking between ICX6610, ICX7450 and VDX6740 (fiber models).

How many ports? A 7450 has 3 module slots, so a max of 3x 40Gb, 12x 10Gb or some combination thereof. Mine is in a rack with an Arista and several servers, so I couldn't tell you how loud it is; it doesn't seem to stick out versus anything else unless it gets into temp range 2 and sets the fans to 100%.

 

[dcplaya]

Does anyone have iPXE booting working?

Currently, I have VLAN 2001 setup on the switch and it has `helper-address` set to my DHCP server (also my router, VyOS).
I can plug in a running device (Windows laptop) and I get a IP address from the correct DHCP. But when I boot a PXE device, it never gets an IP address and doesn't even seem to try!

This is not a LAGG port, just a standard port.

My config can be seen here

Any help would be greatly appreciated!

 

[luks]

How many ports? A 7450 has 3 module slots, so a max of 3x 40Gb, 12x 10Gb or some combination thereof. Mine is in a rack with an Arista and several servers, so I couldn't tell you how loud it is; it doesn't seem to stick out versus anything else unless it gets into temp raneg 2 and sets the fans to 100%.

Two 40Gb and 4x 10Gb would be enough.

ICX6610 seems to have higher power consumption on the datasheet. But it's a lot cheaper to buy

 

[jasonwc]

The spec sheet for the ICX7450-48p shows 106W idle power consumption and 49dB acoustics. The acoustics are basically identical to an ICX6610-48p with dual B/C power supplies and dual fans (48.7dB per spec sheet). As for power consumption, the first post on this thread lists the ICX6610-48p at 110W idle. The spec sheet shows 165W for one PSU and 185w for two, but that doesn’t match real world idle usage.

 

[Wolfcastle]

I just picked up a 7450-24p on ebay and have a 6610-24p on the way, I will be able to tell you soon exactly how they compare!

Does anyone have experience with the ICX7450? How does it compare to the ICX6610? How loud is it? I need a switch that supports both 10Gb and 40Gb. I'm thinking between ICX6610, ICX7450 and VDX6740 (fiber models).

 

[bween]

Hi, hoping someone can help with HomeKit and mDNS.
Setup:
- home assistant with homebridge to homekit, on servers vlan 10
- HomePod mini, on trusted vlan 20
- rpi running avahi-daemon, interface on vlan 10/20
- all routing done

Problem:
When my iPhone (connected to vlan 20) is on the home network, everything works fine. When my iPhone is not connected to the home network, all of the homebridge ONLY devices are not responding. Problem immediately goes away when the iPhone is connected to the home network.

Tried restarting devices, repairing things etc to no avail.

Is there some setting on the switch I need to enable for HomePod to recognize avahi as the mdns reflector? Maybe an ip helper address or something?

 

[richtj99]

I have a Dahua POE IP camera on my 7250-48P. It stopped working & I am trying to troubleshoot.

When I plug it in, the POE does send power - is there a way to see what IP it has through a ssh session?

Not sure if it changed IP's or is working properly or at all. It seems to power up or at least the switch is sending power.

Any ideas on how to get more info from the switch?

SSH@7250#sh inline power
Power Capacity:         Total is 740000 mWatts. Current Free is 354200 mWatts.
Power Allocations:      Requests Honored 3209 times
 Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
 1/1/30 On      On          7300      30000  802.3at  Class 4     3  n/a
--------------------------------------------------------------------------
 Total                     76000     385800

SSH@7250(config)#sh int e 1/1/30
GigabitEthernet1/1/30 is up, line protocol is up
  Port up for 4 minute(s) 39 second(s)
  Hardware is GigabitEthernet, address is 609c.9f75.c9f4 (bia 609c.9f75.ca11)
  Configured speed auto, actual 100Mbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual MDIX
  EEE Feature Disabled
  Untagged member of L2 VLAN 68, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is config enabled, oper enabled, negotiation disabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  VLAN-Mapping is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  IPG MII 0 bits-time, IPG GMII 0 bits-time
  MTU 1500 bytes, encapsulation ethernet
  MMU Mode is Store-and-forward
  300 second input rate: 80 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 49016 bits/sec, 27 packets/sec, 0.04% utilization
  282536433 packets input, 396072306369 bytes, 0 no buffer
  Received 122868 broadcasts, 3887 multicasts, 282409678 unicasts
  12 input errors, 12 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  167684904 packets output, 14766722895 bytes, 0 underruns
  Transmitted 27103983 broadcasts, 5383352 multicasts, 135197569 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled
  Protected: No
  MAC Port Security: Disabled

UC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0           135042966                   0
         1                   0                   0
         2                   0                   0
         3                   0                   0
         4               11274                   0
         5                   7                   0
         6                   0                   0
         7              586058                   0


MC Egress queues:
Queue counters    Queued packets    Dropped Packets
         0             4850420                 141
         1            27020055                   0
         2              170042                   0
         3                4087                   0

SSH@Office7250(config)#

 

[LodeRunner]

Hi, hoping someone can help with HomeKit and mDNS.
Setup:
- home assistant with homebridge to homekit, on servers vlan 10
- HomePod mini, on trusted vlan 20
- rpi running avahi-daemon, interface on vlan 10/20
- all routing done

Problem:
When my iPhone (connected to vlan 20) is on the home network, everything works fine. When my iPhone is not connected to the home network, all of the homebridge ONLY devices are not responding. Problem immediately goes away when the iPhone is connected to the home network.

Tried restarting devices, repairing things etc to no avail.

Is there some setting on the switch I need to enable for HomePod to recognize avahi as the mdns reflector? Maybe an ip helper address or something?

Is the reflector enabled in your Avahi config?

Edit: to be honest, this is one of the reasons I flattened my network.

 

[bween]

Is the reflector enabled in your Avahi config?

Edit: to be honest, this is one of the reasons I flattened my network.

Thanks for the quick reply @LodeRunner.
Yes, reflector is enabled with “enable-reflector=yes” in /etc/avahi/avahi-daemon.conf. No other configuration edited from default though.

And yes, this issue is driving me nuts… seems like most guides/instructables out there suggest enabling reflector in opnsense or something but I’d like to get it working on the brocade if possible

 

[LodeRunner]

All I can say is good luck. I got avahi mDNS reflector working on a VM with vNICs interfaces in both VLANs (no tagging or sub-/virtual interface in the VM). It worked about 75% of the time.

And you have properly tagged VLAN sub-interfaces configured on the rPi? And those interfaces are passing traffic? And there's no default firewall on the rPi?

 

[kpfleming]

Edit: to be honest, this is one of the reasons I flattened my network.

Ditto. Reduce the number of VLANs, and use access-lists to control access between devices. The world is much simpler that way.

 

[bween]

All I can say is good luck. I got avahi mDNS reflector working on a VM with vNICs interfaces in both VLANs (no tagging or sub-/virtual interface in the VM). It worked about 75% of the time.

And you have properly tagged VLAN sub-interfaces configured on the rPi? And those interfaces are passing traffic? And there's no default firewall on the rPi?

Yep, VLAN sub-interfaces seem to be working properly. Am able to ssh and connect to other devices through both interfaces on both VLANs. Default firewall is off. Did you ever figure out why it worked only sporadically?

Would you mind sharing more details on what you're doing now instead? Seems like bonjour and avahi might just be a headache not worth going deeper into.

 

[jasonwc]

Is the reflector enabled in your Avahi config?

Edit: to be honest, this is one of the reasons I flattened my network.

Fortunately, Avahi is dead easy to setup on pfsense. You just select the interfaces you want the mdns packets to be reflected on and then check one box to enable repeating mdns packets across subnets. Once that's configured, I was able to control HomeKit devices across subnets without issues. I suspect the same can be done on the Brocade switch but I see no reason to fix what's already working.