Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. svtkobra7

    svtkobra7 Active Member

    Joined:
    Jan 2, 2017
    Messages:
    324
    Likes Received:
    65
    Just take one from this guy's book ...

    [​IMG]
     
    #1001
    fohdeesha likes this.
  2. mattaw

    mattaw Member

    Joined:
    Jul 30, 2018
    Messages:
    40
    Likes Received:
    13
    Nice purchase! Join the lucky club as I got 3x for $160 icx6610 48P's.

    Does anyone have a scrap mainboard? I need two replacement push pins for the main routing IC heatsinks. I guess they failed with heat and age.

    IMG_20181109_223624.jpg

    I'm also getting a poe PD error on port 48. It powers fine, but won't sense current etc. nor light the poe led. Swapped the Poe board into another switch and the problem followed the board. Just scrubbed the board as it was dirty but all the components look ok to me. Thoughts?
     
    #1002
    Last edited: Nov 9, 2018
    fohdeesha likes this.
  3. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,488
    Likes Received:
    1,253
    never seen those pins fail, crazy. Wonder if it got knocked around? I have a junk ICX6650, it might have the same heatsinks/pins from the looks of it, I'll take a look.

    Wanna do me a huge favor, if the heatsink is free now, can you move it to the side and get the text/model number of the ASIC? They've kept it incredibly secret - I know it's a Marvell Cheetah 5, which I'm pretty sure was a custom build just for Brocade, but it would be nice to have some verification/ actual P/N

    in my experience, PD errors that follow the board on a specific port mean that port took a pretty nasty indirect or direct static hit (usually by a PoE device like a camera that was outdoors). No way to fix that I know of
     
    #1003
  4. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,488
    Likes Received:
    1,253
    no luck on the 6650 heatsink, it's metal bolts to the chassis. There's pushpins for the mgmt CPU next to it, but they look much smaller than what you need. You'll probably just have to do what I did for some old blade heatsinks, go to the hardware store and find some matching nuts, bolts and washers. stick the bolt up through the bottom of the PCB so you don't have to worry about clearance underneath (any extra bolt length will just be sticking up through the heatsink) just don't over tighten them, they barely need to be snug

    https://i.imgur.com/meDUqfv.jpg
     
    #1004
  5. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,488
    Likes Received:
    1,253
    If anyone is interested: I have reverse airflow (Intake) PSU's (2) and fan trays (2) I'm looking to trade for the regular exhaust models. Pretty sure they'd all fit in a medium or large USPS flat rate box, so maybe $12 shipping for each of us

    EDIT: trade has been made, no longer have them
     
    #1005
    Last edited: May 22, 2019
    Emanuele and tommybackeast like this.
  6. mattaw

    mattaw Member

    Joined:
    Jul 30, 2018
    Messages:
    40
    Likes Received:
    13
    Thanks for the info, EOS on the port sounds very likely. I should look up the chip that supplies the power and see what the filter design is.

    As for the heatsinks they seem glued onto the ics. It may just be ancient thermal grease? Have you tried freezing them off - it was an old trick for GeForce cards.
     
    #1006
  7. Juggie

    Juggie New Member

    Joined:
    Nov 3, 2018
    Messages:
    20
    Likes Received:
    6
    Are these brocades able to do sfp (not sfp+) in 2.5gbit? The local fiber provider here uses sfp gpon at 2.5. it works with their crappy modem and also with a ES-16-XG from ubiquiti which has recently added it. Folks have tried with other gear including Intel sfp cards but they all seem to be limited to 1gig on sfp.
     
    #1007
  8. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,488
    Likes Received:
    1,253
    I would think it's just caked up thermal compound - I can't imagine why they would use thermal adhesive AND retaining pins. Can you try to gently rotate the heatsink to break it free? (don't pull up). I've never personally attempted removing them, since the retaining pins are one time use (once you compress them to get them out of the hole, they never quite work right again)
     
    #1008
  9. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,488
    Likes Received:
    1,253
    nope, it will link at 1gbe
     
    #1009
  10. Juggie

    Juggie New Member

    Joined:
    Nov 3, 2018
    Messages:
    20
    Likes Received:
    6
    K so no 2.5 or 5 gbit ethernet in these.
     
    #1010
  11. mattaw

    mattaw Member

    Joined:
    Jul 30, 2018
    Messages:
    40
    Likes Received:
    13
    Please accept a weeks delay - I have a conference
     
    #1011
    tommybackeast and fohdeesha like this.
  12. maes

    maes Member

    Joined:
    Nov 11, 2018
    Messages:
    52
    Likes Received:
    27
    Longtime lurker here, finally decided to take the jump and go for an ICX6450-24 for the home lab. It'll be a nice boost for the NAS that's running over direct-attach 10Gbe right now.

    I'm having the worst of lucks finding one with rackmount tabs on, that will ship outside of the USA and is still affordable, but found a few tab-less ones in the $150-$170 range.
    Anyone would have the p/n for the accessory kit or compatible replacement rackmount tabs?
     
    #1012
  13. mattaw

    mattaw Member

    Joined:
    Jul 30, 2018
    Messages:
    40
    Likes Received:
    13
    Luckily brocade part numbers are sane: ICX6400-RMK
     
    #1013
    tommybackeast, fohdeesha and maes like this.
  14. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,488
    Likes Received:
    1,253
    #1014
  15. maes

    maes Member

    Joined:
    Nov 11, 2018
    Messages:
    52
    Likes Received:
    27
    Thanks, I actually found it at two local stores for a good bit lower, especially with shipping taken into account. The eBay link is asking for ~$75 for international shipping.

    Channel Electronics - Buy Mfgs Part-#:ICX6400-RMK online in Canada
    Ruckus 2 Post - network device mounting kit - ICX6400-RMK - Rack Mounting Accessories - CDW.CA

    I'll see when I get the actual switch and confirm it runs. In the meantime, back to modding my NSC-800!

    Oh do you still have any of those 10gbe licenses by any chance?
     
    #1015
    Last edited: Nov 12, 2018
  16. Cutha

    Cutha Member

    Joined:
    Sep 24, 2016
    Messages:
    57
    Likes Received:
    6
    Thanks for all the fantastic information @fohdeesha ! It saved me from buying a pair of MicroTik's.

    My kids computers are in the basement and when I seriously started down this rabbit hole my server rack was was a rigged up 2 post with 2x4's for rear supports and a couple of comparatively quiet servers and a switch. It is now a full size rack with multiple Supermicro servers and an Infiniband switch. The heat and noise proved to much and everything is being relocated. I was trying to figure out how to bridge between the workstations and the server without sacrificing bandwidth when I saw this post and I have now purchased 2* "Brocade ICX 6610-48P 48-port 1 GbE RJ45 PoE+ Ethernet Switch ICX6610-48P".

    I plan on using a "Mellanox FDR 4X Infiniband& 40G Ethernet QSFP+ 15M cable MC2207310-015" to connect the two ICX6610's. Will that work?

    I could use a bit of help figuring out what to purchase to finish this up. The workstations had 1GB Cat6 connecting them to the regular network and 40GB IB connecting them to the storage on the servers. I have a bunch of Mellanox MHQH29B-XTR (flashed for RDMA) and varied lengths of QDR QSFP DAC's. What is going to be the cheapest route to connect 6 workstations to the 10GB and 40GB ports? I think the Mellanox cards I have are IB only.

    The cable run for the workstations to the switch are between 15' and 30'. I am assuming that I will need to purchase new cabling and cards. My current thought is buy 4 cheap Mellanox EN cards and use a QSFP to 4x10GB SFP+ cable and for the other two workstations get 2 Mellanox 40GBE cards and use my existing QSFP DAC cables. Does this make sense? Is there a cheaper way? Any suggestion for NIC part numbers to search for?

    Thanks again for all the great information.
     
    #1016
    fohdeesha likes this.
  17. PnoT

    PnoT Active Member

    Joined:
    Mar 1, 2015
    Messages:
    584
    Likes Received:
    119
    I got my ICX6610-48P in today and it showed up with 2 PSU and 2 FANs that are reverse airflow! It's time to start setting this thing up.
     
    #1017
    Last edited: Nov 12, 2018
    fohdeesha likes this.
  18. Juggie

    Juggie New Member

    Joined:
    Nov 3, 2018
    Messages:
    20
    Likes Received:
    6
    How many folks am I dogging it out with on Ebay trying to get a hold of a 6610/6450-48P? :) Seems to be slim pickings at the moment at least those that are good deals.
     
    #1018
  19. Skud

    Skud Member

    Joined:
    Jan 3, 2012
    Messages:
    71
    Likes Received:
    18
    Got my 7250-24P today and initially I powered it up without the console plugged in. This unit takes way longer to boot than the 6610 and I thought it was DOA since the fans weren't coming down.

    I built a mini-usb to RJ45 so I could use my standard Cisco cable and then I saw that it was in fact booting, it just sits at "Starting Kernel" for a long time before moving along..

    Here is the pinout for the contraption I made. Seems to be working well.
    Code:
    Mini-USB           Keystone (568B)
    ========            ===============
    1 (VCC)            N/C
    2 (UART RX)        6 (Grn)
    3 (UART TX)        3 (Grn/Wht)
    4 (Reserved)       N/C
    5 (GND)            4 (Blue)
    
    Off to tinker!!

    Thanks!!
    Riley
     
    #1019
    Lambdariver, maes and fohdeesha like this.
  20. koisama

    koisama New Member

    Joined:
    Oct 22, 2018
    Messages:
    7
    Likes Received:
    0
    Is $300 a reasonable price for non-poe 6610-24P with two power supplies and fans?
    Offered $250, seller won't go below $300.
     
    #1020
Similar Threads: Brocade Series
Forum Title Date
Networking Mikrotik compatible with Brocade 57-0000075-01 transceivers? Dec 17, 2019
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019

Share This Page