Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. HomelabN0rth

    HomelabN0rth New Member

    Joined:
    Jan 30, 2019
    Messages:
    3
    Likes Received:
    2
    Oh sorry, I was in TERMINALDAD> mode - I wasn't going to let it "win", ya know? I have to admit, the documentation from Rukus is what saved the day. It wasn't all where I was looking for it but some Google-fu with the error messages and commands was enough to bring up pages where they posted how to do each individual part, even if each part was on a separate document LOL
     
    #1901
    fohdeesha likes this.
  2. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,389
    Likes Received:
    1,117
    :p I will say you got lucky, half of the "development" switches I have seen with these bootloaders have an "update_boot" routine that lays the bootloader file down unmodified at the beginning of flash (which worked for whatever development bootloaders they were using). However the "production" bootloader files that we all have have around 30 lines or so of metadata at the beginning that needs to be stripped off first, if you flash those using the update_boot of a dev bootloader and it lays it down raw, it'll brick it. It sounds like yours already had a production bootloader though, that just happened to be in diag mode with a bunch of strange boot environment variables set for the broadcom image (similar to @britinpdx ). I almost want to ask g-electronic where he is getting these because they keep coming with the strangest stuff I've seen
     
    #1902
    Last edited: Jun 6, 2019
    HomelabN0rth likes this.
  3. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,389
    Likes Received:
    1,117
    Nice find. after skimming through the patch files some more it turns out the secret mode is still there, they just renamed the routine and environment variable.

    for the ICX6450:
    (it also requires setting of pp_init for this mode to occur)

    setenv pp_init_enable 1
    setenv diag_cmd_on 1
    saveenv
    reset

    ##go back to normal
    setenv pp_init_enable
    setenv diag_cmd_on
    saveenv
    reset


    For the ICX7650:

    setenv diag_mode_on 1
    saveenv
    reset

    ##go back to normal
    setenv diag_mode_on
    saveenv
    reset
     
    #1903
    sean likes this.
  4. Skippy96

    Skippy96 New Member

    Joined:
    May 14, 2019
    Messages:
    14
    Likes Received:
    5
    Is there any way to leverage all the access we have with these dev builds to modify the firmware to allow for fan modifications?
    My 6610 is getting too loud for my office :)
     
    #1904
  5. sean

    sean Member

    Joined:
    Sep 26, 2013
    Messages:
    58
    Likes Received:
    29
    Nice. I checked the patch files but they're big and I missed anything obvious. I now see the pp_init_enable and hilariously a comment about hiding the diag commands.

    I defer to fohdeesha but for me, the other models using u-boot and Linux means Arris has to release their modifications. It's then "easy" (the u-boot patch is 400k lines) to go through their changes and find stuff they've hidden. In the case of Linux, it's easier to take their firmware images and open them to see the surrounding environment and how it boots. Actual hardware control however is still limited to FastIron and their kernel module. Those are proprietary and much harder to inspect, much less modify.

    The 6610 has none of this however. Someone else was working on a way to lie about the fan RPMs and I think that will be the quickest/only way to modify the fans.
     
    #1905
  6. spali

    spali Member

    Joined:
    Nov 4, 2018
    Messages:
    32
    Likes Received:
    3
    Thought a second time about it.
    10g downlinks wouldn't be a bottleneck, due i will not serve higher bandwidth from the 6610.
    Can i use a standard QSFP DAC in the breakout ports to uplink to an arista QSFP port just set to breakout mode? Kind of breakout to breakout?

    Second question... My two 6110's I ordered each a single Rev. A PSU ... I have read the whole thread now and because they seem not to spin down when in "single mode" and because the intake version RPS16-I is really rare to find. I would like to ask if anyone which has a lot of equipment could do a test for me.
    I'm thinking if it could work to put as second a non POE together with a POE PSU in each switch to let them spin down. If this would work I would just not have the redundancy in POE?
    Hope explained it halfway understandable :rolleyes:
     
    #1906
  7. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,389
    Likes Received:
    1,117

    like Sean said, on either the linux based models, or the non-linux based models, both of them use a proprietary module/binary to control the fans once booted, so nothing we can do there. you can control them all you want from the bootloader, but as soon as the fastiron app boots it takes over chassis control
     
    #1907
  8. Emanuele

    Emanuele New Member

    Joined:
    Mar 1, 2019
    Messages:
    16
    Likes Received:
    1
    Hi all! Thank to @kiteboarder I landed to this thread, @fohdeesha great work here :) ,and after a couple of pages I decided to go all in for 2x ICX 6610-24 (non poe version).
    @fohdeesha great work
    One of them arrived without rack ears, I read here where to find a set (they cost a lot BTW).
    The problem has presented the moment I turned it on.
    It has INTAKE FAN and EXHAUST PSU, so it is returning a critical error and reboot.
    IMG_20190608_130515.jpg
    Again and again obviously.
    IMG_20190608_125719.jpg
    So, I can't play with my new toy :(
    I wrote down to the seller but I don't think he has lots of spare parts of this switch.

    Does anyone here is willing to sell 4x INTAKE PSU to EU (Italy)? Maybe not too much €$£?
     
    #1908
  9. spali

    spali Member

    Joined:
    Nov 4, 2018
    Messages:
    32
    Likes Received:
    3
    Searched a lot in the last days, they are almost not available in Europe for a descent price.
    But if switching airflow to exhaust is an option, I would get new FAN's. There are currently available for 100$ inc. shipping to Europe on ebay.
     
    #1909
  10. Emanuele

    Emanuele New Member

    Joined:
    Mar 1, 2019
    Messages:
    16
    Likes Received:
    1
    :eek: I did pay 480€ inc. shipping per the 2x switch, I can't afford 100$ per a single fan module... it's way overpriced...
     
    #1910
  11. spali

    spali Member

    Joined:
    Nov 4, 2018
    Messages:
    32
    Likes Received:
    3
    I know, the prices in Europe are horrible and most times shipping from US is cheaper even the shipping costs are height. Just ordered two intake FANs from them, because my two 6610-48P-I that are on the way, have only one FAN and one PSU. PSU is also Rev. A :(
    But was the best deal I found in the last time for 400$ inc shipping each. And I'm hopping to find some intake PSUs in the future. But mine work at least, but loud. But worst case would be for me to stack and just shut one of them down and put the PSU into the other to let the PSU fans spin down.
    In your case it would be probably best to search for a good deal of any 6610 where you just take the fans out. Would be cheaper than buying them extra. Except someone in europe has some spare parts.
     
    #1911
  12. Emanuele

    Emanuele New Member

    Joined:
    Mar 1, 2019
    Messages:
    16
    Likes Received:
    1
    You are probably right, prices can't be helped. I'll start looking around for spare parts/broken 6610, if someone want to sell something, here I am :)
     
    #1912
  13. spali

    spali Member

    Joined:
    Nov 4, 2018
    Messages:
    32
    Likes Received:
    3
    I queue up behind @Emanuele for two RPS16-I
    :D
     
    #1913
  14. arglebargle

    arglebargle H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

    Joined:
    Jul 15, 2018
    Messages:
    634
    Likes Received:
    208
    I mean, you could order a switch from g-electronic and dump them then return it as "faulty"

    #JustSayin'

    RE Rack ears: I think we've hit the point where modelling rack ears for the various switches would be a smart move so we can print our own and skip the $50-60 tax every time a switch ships without them. I'm eyeballing a couple of 3d printers at the moment, if no one else bites I'll do the work in the next couple of months if I continue to have free time like I do right now.

    If anyone has experience doing simple static load modelling and can point me at some basic resources for designing durable parts I'd appreciate it.
     
    #1914
    Last edited: Jun 9, 2019
    tommybackeast and ewer0012 like this.
  15. Hakujou

    Hakujou New Member

    Joined:
    Apr 28, 2019
    Messages:
    17
    Likes Received:
    6
    While g-electronic has mostly bad switches and they probably just test them by checking they power on (to be fair, they don't claim otherwise), their customer service has always been honest and refunded everyone without trouble (they even let me kept my faulty switch while refunding me entirely).

    That kind of behavior is just dishonest and could lead to a more troublesome customer service.
     
    #1915
    yukaia and Emanuele like this.
  16. ewer0012

    ewer0012 Member

    Joined:
    Feb 10, 2019
    Messages:
    58
    Likes Received:
    22
    I managed to pick up an ICX6450 for $100 and an ICX7250 for $150 on Friday. Hoping these will do the trick so I can get away from MikroTik.

    Thanks for sharing, OP.
     
    #1916
    FRPII likes this.
  17. BeTeP

    BeTeP Active Member

    Joined:
    Mar 23, 2019
    Messages:
    242
    Likes Received:
    97
    Are we still talking about 2 small pieces of angled steel with a few holes? I just grab ears from any old piece of equipment and drill the missing mounting holes if any. If you don't have any laying around - Cisco 1U rack ears sell for like $5/pair shipped from China.
     
    #1917
  18. arglebargle

    arglebargle H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

    Joined:
    Jul 15, 2018
    Messages:
    634
    Likes Received:
    208
    Yeah, we are. Rack ears are one of those things that are ludicrously priced when you want the actual OEM part. Being able to 3d-print them out of like $10 worth of nylon would be kind of nice.
     
    #1918
  19. cuco

    cuco Member

    Joined:
    Feb 13, 2018
    Messages:
    39
    Likes Received:
    1
    I would also appreciate it and go all in
     
    #1919
  20. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,418
    Likes Received:
    358
    I don't have any experience with 3D printing but not sure if the material used for making the brackets has enough strength to withstand this application especially for 6610. The rear of the switch sags due to the weight. Imagine the twist excerted even on the OEM metal brackets. It may be ok for 6450.
     
    #1920
    Last edited: Jun 10, 2019
Similar Threads: Brocade Series
Forum Title Date
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019
Networking Brocade VDX 6720 - what do I need to know? Sep 16, 2019

Share This Page