Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I finally thought my ICX6610 was on it's way out the door because I just picked up 7250-48p BUT...
I'm guessing this is a bad POE module?
Picked it up from ready2use @ ebay
Code:
PoE: Stack unit 1 PS 1, Internal Power supply with 740000 mwatts capacity is up
PoE Info: Adding new 54V capacity of 740000 mW, total capacity is 740000, total free capacity is 740000
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
2000 log entries of PoE Event Trace Log Buffer is allocated on unit 1 for unit 1
PoE Event Trace Logging enabled for unit 1
PoE Error: Failed Devices Bitmap on PoE module:3f, numDev:6.
Resetting module in slot 1 again to recover from dev fault
PoE Info: Hard Resetting in slot 1....
PoE Info: Resetting module in slot 1....completed.
PoE Error: Failed Devices Bitmap on PoE module:3f, numDev:6.
PoE Severe Error: Hardware Fault with ports 1/1/1 to 1/1/8. Remove PDs and then configure "no inline power" on these ports.
PoE Severe Error: Hardware Fault with ports 1/1/9 to 1/1/16. Remove PDs and then configure "no inline power" on these ports.
PoE Severe Error: Hardware Fault with ports 1/1/17 to 1/1/24. Remove PDs and then configure "no inline power" on these ports.
PoE Severe Error: Hardware Fault with ports 1/1/25 to 1/1/32. Remove PDs and then configure "no inline power" on these ports.
PoE Severe Error: Hardware Fault with ports 1/1/33 to 1/1/40. Remove PDs and then configure "no inline power" on these ports.
PoE Severe Error: Hardware Fault with ports 1/1/41 to 1/1/48. Remove PDs and then configure "no inline power" on these ports.
PoE Info: PoE module 1 of Unit 1 initialization is done.Picked it up from ready2use @ ebay
I actually went through this on my first ICX6610-48P having a bad module so this is just nuts how these sellers don't bother to check the switches before selling them as "100% working".It definitely looks like a bad PoE board...
Not to justify them, but I'd want to bet on that most of those sellers don't even have a console cable.
They might just clean the dust, power it on holding reset for 30s, plug a cable on port 1 and another on any other ports and see if the device got a link UP and maybe a DHCP address.
And to be fair, reselling a bunch of used enterprise gear, of a ton of different vendors, with different console cable, different GUI and CLI, and know about all the messages that it spews out during boot... they'd be consultants, not sellers.
Most of the times (for me anyway), it does matter more how a seller react to a bad item, that receiving a bad item in itself... take for granted that $hit happens, now what? If the seller accepts to exchange or refund, I'd not be throwing rocks at him.
Justify them you did... you sir sound like a seller!
This is exactly the type of stuff that shady sellers get away with because of people like you making excuses for them. "100% tested" / "100% working" means something different to me I guess. What about my time and money that's completely wasted because a seller can't be bothered to know enough about his product to buy a cable and actually use it? A seller will demand a higher price for a POE switch but it's ok for them to not test POE...
Anyway, hopefully I can find one that works and so the search begins again.
I moved the hdhomerun and a laptop over to a dumb switch, works.
Super weird that some multicast is working (ie chromecast's are still working!) but the hdhomerun is not.
Of interest I can also see the hdhomerun beacon out (ie windows sees it and adds it to 'networking'. And I can see perdiodic multicast messages while dumping packets, but if you send a query over multicast, it doesn't reply. It's as if messages are getting out, but not in.
If anyone has any suggestions i'd love to hear them, i am all out of ideas.
I lack the skill to add anything to the wiki; but wanted to thank you for starting it....
Take care of yourself.................I have still been away dealing with medical issues but I am glad to see other members stepping in to help others
doing my best to get to license requests but it may take a couple days so don't worry if I don't reply immediately
I searched on their ICX forums and someone had issues with multicast on a 7250. They found that going from 8.0.30r to 8.0.30h fixed their issue. Not sure if that will correct it but multicast is one of those things I don't do very much and feels a bit like black magic. You should also take a look at the 8.0.30 manual from Ruckus for IP Multicast Guide. You may want to play around with tracking, fast leave, fast convergence options.
No sir, I can assure you that I've never sold anything.
I just said that most of these products come from sellers that resold discarded enterprise hardware... It just isn't feasible for them to know about any quirks and pitfalls of all the different models of hardware that they sell. I 100% agree with you that they shouldn't say 100% tested or something similar, but that's the way it is for most of the used things.
The reality of this world is just one: do you want reliability and warranty? Pay the premium and buy new. There is a reason if those units were sold for a fraction of the list price; heck, it could even be working and break the next time you power it on, I've seen plenty of perfectly working units, powered on for years, give up the ghost after a power cycle.
And trust me, the last thing I want in my life is waste time and money, but at the same time I can't fight the war by myself, so... this is life.
upgrading from 30t -> 30u solved the problem. Hat tip to a post I found on reddit here: I sent the guy a pm and he let me know 30u solved the issue. Flashed that, issue fixed.
I'm wondering if someone can help me with my IPv6 setup with my new ICX 6610. I currently have a Mikrotik RB4011 as my WAN gateway/firewall. It provides DHCP for VLANs (VLAN 10 & 20) behind my Brocade that don't have a presence on the Mikrotik. My ip `helper-address` relays DHCP requests to the firewall. It works exactly as expected. My Brocade is able to do all the switching and forward requests to the firewall via a static route.
I would like something similar to happen with IPv6, but I'm unsure how to get it to happen. I am able to get a /60 prefix from my ISP and I'd like to give one of each of the /64 prefixes to a different VLAN behind the Brocade. I don't know how to do that programmatically, so right now I have them hardcoded in the brocade. I choose a different subnet for each VLAN, then have a static route setup back to the firewall. It works, each of my IPv6 clients configure their own address on their own IPv6 subnet. This is the relevant config section:
ipv6 route ::/0 ve 250 fe80::XXXX:XXff:feXX:XXXX
!
interface ve 10
ip address 172.16.10.1 255.255.255.0
ip helper-address 1 10.250.250.1
ipv6 address 2600:XXXX:XXXX:XXX1::1/64
ipv6 enable
ipv6 nd prefix-advertisement 2600:XXXX:XXXX:XXX1::/64 1000 800 onlink autoconfig
!
interface ve 20
ip address 172.16.20.1 255.255.255.0
ip helper-address 1 10.250.250.1
ipv6 address 2600:XXXX:XXXX:XXX2::1/64
ipv6 enable
ipv6 nd prefix-advertisement 2600:XXXX:XXXX:XXX2::/64 1000 800 onlink autoconfig
!
It works fine, and my prefix doesn't seem to change very often (I've never tracked it very carefully, and I was previously using pfSense with 1 VLAN so I wouldn't have noticed if it did change, but AFAIK, it's never changed) so it isn't that big a deal to hard code them, but I'd love if there was a way to do it programmatically. Could anyone tell me if/how I can get my prefix programmatically? My understanding is that I need to run a DHCPv6 server on my Mikrotik, but I'm not sure what more to do beyond that. The documentation for ipv6 dhcp-relay makes it sound like something used for client DHCPv6 requests, which isn't what I want. I want my clients to continue to auto-config and just send the prefix to the Brocade.
I would like something similar to happen with IPv6, but I'm unsure how to get it to happen. I am able to get a /60 prefix from my ISP and I'd like to give one of each of the /64 prefixes to a different VLAN behind the Brocade. I don't know how to do that programmatically, so right now I have them hardcoded in the brocade. I choose a different subnet for each VLAN, then have a static route setup back to the firewall. It works, each of my IPv6 clients configure their own address on their own IPv6 subnet. This is the relevant config section:
ipv6 route ::/0 ve 250 fe80::XXXX:XXff:feXX:XXXX
!
interface ve 10
ip address 172.16.10.1 255.255.255.0
ip helper-address 1 10.250.250.1
ipv6 address 2600:XXXX:XXXX:XXX1::1/64
ipv6 enable
ipv6 nd prefix-advertisement 2600:XXXX:XXXX:XXX1::/64 1000 800 onlink autoconfig
!
interface ve 20
ip address 172.16.20.1 255.255.255.0
ip helper-address 1 10.250.250.1
ipv6 address 2600:XXXX:XXXX:XXX2::1/64
ipv6 enable
ipv6 nd prefix-advertisement 2600:XXXX:XXXX:XXX2::/64 1000 800 onlink autoconfig
!
It works fine, and my prefix doesn't seem to change very often (I've never tracked it very carefully, and I was previously using pfSense with 1 VLAN so I wouldn't have noticed if it did change, but AFAIK, it's never changed) so it isn't that big a deal to hard code them, but I'd love if there was a way to do it programmatically. Could anyone tell me if/how I can get my prefix programmatically? My understanding is that I need to run a DHCPv6 server on my Mikrotik, but I'm not sure what more to do beyond that. The documentation for ipv6 dhcp-relay makes it sound like something used for client DHCPv6 requests, which isn't what I want. I want my clients to continue to auto-config and just send the prefix to the Brocade.
No worries. It was a tangly issue but no fault of yours! I had broken out wireshark, found a python example to dump out multicast, and subbed to the ip/port used by the hdhomerun (which is ssdp). I was seeing all kinds of traffic including the hdhomerun itself broadcasting out, it just never replied when it was asked 'hey are you there'. Everything else ie chromecast and the like was all working. When i switched it over to a dumb switch and it worked i then knew for sure it was a firmware bug. I had it on my list to try latest firmware and latest switch firmware (since I don't use the routing features) to see if either of those worked before I came across the reddit post. I was surprised to see a bug of that nature in fairly mature firmware but, I guess it was it was a regression introduced somewhere along the way.
Appreciate all you've done here. I am very happy with my 6610-48p. It was a bit loud at first, not because it was loud itself (it's in the basement near the furnace which is obviously louder!), but because it was on a 1U shelf (i don't have rails) and it was causing vibration, that reverberated from the wall mounted rack, to the 2x4 and I could hear it upstairs at night when it was quiet. I added some micro-fiber feet (a small version of the sticky things you get to protect hardwood floor from furniture) problem solved, mostly. I also have some rubber washers I picked up last week to redo how the rack is mounted on the wall and that will eliminate the chance of it occurring entirely.
One thing I'll add is i got a few suggestions here to play with multicast/igmp settings on the switch. I did, but as far as I can tell, those have zero impact on vlan1 (the default vlan). Once you enable it, ie 'ip multicast active' any of the 'show ip multicast *' commands all show vlans, but never vlan 1. I suspect vlan1 was designed to be broadcast to all ports only, perhaps you can confirm fohdeesha? I also was going to try the hdhomerun on a vlan to see if the switch would detect it and show it in the igmp status, but i upgraded the firmware first which fixed it and that wasn't necessairy.
I recently picked up an ICX7150-C12P on Ebay. It managed to boot once and then all it would do is flash the leds once when you plugged it in and that was about it. Found the reddit thread recommending this PS SL Power LB130S56K 56V 2.32 130W Power Supply 689192481516 | eBay. FYI they don't go for 30 bucks any more
After receiving the replacement supply and removing the old supply, I slid off the plastic mounting retention from the pins (they are tight but it can be done) and flipped them so the output cables were correct. I also took the old AC input retention plastic and installed it on the new power supply so the ac input cable fit correctly.
This fixed the booting issue. After booting with installed firmware 08060d the console log continuously spewed POE power errors. Since this is still the non-ufi based firmware, I had to upgrade to the non-ufi 08090b first and install the new bootloader. After a reboot the POE firmware was upgraded automatically. Error messages were now gone. I then upgraded to the ufi version of 08090b and rebooted. This allowed me to install the latest 08092b ufi only firmware and the 7150 has been working great the last couple of days.
Looking at the original power supply, it seems heat dissipation was an afterthought with all the conductive thermal pads added along with all the shrink wrapped metal bars that were on top of the mosfets(didn't look up the part number so don't quote me on this) to increase contact to the aluminum frame of the original power suppply. I have actually never seen thermal pads this thick before. Really bad design.
Thanks to this forum and @fohdeesha 's posts on these switches which are a hell of a deal.
--MikeP
After receiving the replacement supply and removing the old supply, I slid off the plastic mounting retention from the pins (they are tight but it can be done) and flipped them so the output cables were correct. I also took the old AC input retention plastic and installed it on the new power supply so the ac input cable fit correctly.
This fixed the booting issue. After booting with installed firmware 08060d the console log continuously spewed POE power errors. Since this is still the non-ufi based firmware, I had to upgrade to the non-ufi 08090b first and install the new bootloader. After a reboot the POE firmware was upgraded automatically. Error messages were now gone. I then upgraded to the ufi version of 08090b and rebooted. This allowed me to install the latest 08092b ufi only firmware and the 7150 has been working great the last couple of days.
Looking at the original power supply, it seems heat dissipation was an afterthought with all the conductive thermal pads added along with all the shrink wrapped metal bars that were on top of the mosfets(didn't look up the part number so don't quote me on this) to increase contact to the aluminum frame of the original power suppply. I have actually never seen thermal pads this thick before. Really bad design.
Thanks to this forum and @fohdeesha 's posts on these switches which are a hell of a deal.
--MikeP
I tried to go down this route as well with my setup. I also get a /60 prefix from my ISP (AT&T) and currently have my edgerouter 4 delegating prefixes to all my VLANS. I posted about his previously in this thread but never got any real direction on how to get the ICX to handle prefix delegation. If you want to read through that conversation you can start at post #3323.
Saw an eBay listing for an ICX-6610-48-PE for $150 with "premium s/w license" but only 1GbE on the SFP 10GbE modules. I offered $120, the seller countered with $140 which I accepted. Sadly the seller said he doesn't see many Rev B or C PSUs, most of them are Rev A it seems. It will be here by end of the week, and will be my first foray into VLANs, fabric/aggregate switches and what not.
I've had to read (most) of the thread before making up my mind and kudos to @fohdeesha and everyone else for an amazing guide + all the hard work.
I've had to read (most) of the thread before making up my mind and kudos to @fohdeesha and everyone else for an amazing guide + all the hard work.
Issues with 4x breakout cables... I think... Well.. at 187 pages.. I'm pretty sure I missed a ton, but my search turned up nothing.
I have two 40G (4x10G) breakout cables, one is cisco (copper), one is arista (fiber) , but they both exhibit the same problem. The line will drop, and not come back up. It is random it seems.
I picked up a new (to me) DL360pG8 which I dropped in an HP branded intel Flexlom card. I originally had 4 10GB links on one of my servers, all were up and functional. I moved 2 of the links to the new server, and the nic lights come on showing connection.. then it drops and the switch shows the port as down. putting the link back onto the original server does not bring the line up......
This is driving me nuts..... I am temped to dump the 40G 4x10G cables, but they are convenient.. and well, if that is not the problem I would have just thrown away some cash which I would prefer not to do..
Version details below
I have two 40G (4x10G) breakout cables, one is cisco (copper), one is arista (fiber) , but they both exhibit the same problem. The line will drop, and not come back up. It is random it seems.
I picked up a new (to me) DL360pG8 which I dropped in an HP branded intel Flexlom card. I originally had 4 10GB links on one of my servers, all were up and functional. I moved 2 of the links to the new server, and the nic lights come on showing connection.. then it drops and the switch shows the port as down. putting the link back onto the original server does not bring the line up......
This is driving me nuts..... I am temped to dump the 40G 4x10G cables, but they are convenient.. and well, if that is not the problem I would have just thrown away some cash which I would prefer not to do..
Version details below
Code:
UNIT 1: compiled on Aug 31 2018 at 07:07:11 labeled as FCXR08030sa
(10539398 bytes) from Primary FCXR08030sa.bin
SW: Version 08.0.30saT7f3
UNIT 2: compiled on Aug 31 2018 at 07:07:11 labeled as FCXR08030sa
(10539398 bytes) from Primary FCXR08030sa.bin
SW: Version 08.0.30saT7f3
Boot-Monitor Image size = 370695, Version:10.1.00T7f5 (grz10100)
HW: Stackable ICX6610-24-HPOE