Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. magi

    magi New Member

    Joined:
    Aug 22, 2019
    Messages:
    8
    Likes Received:
    3
    Ah, forgot to mention that I added the 120mm fans and the temp is well below threshold (40C). Just want to further push down on the thermal equilibrium :)
     
    #2481
  2. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    ah, in that case with those extra fans I think it would take a house fire to get it up to the 80C+ required to hit speed 2
     
    #2482
    magi and Renat like this.
  3. French Chamallow

    French Chamallow New Member

    Joined:
    Mar 15, 2019
    Messages:
    17
    Likes Received:
    3
    Can you tell me if I'm going on the right road :)
    my original language is not english but you must have noticed it ...
    Actually it's my pfsense that manages all the routing.

    that look like that in pfsense :
    Capture


    I have a nano HD AP with several different ssd and 1 vlan per ssid
    I did my first vlan 2 weeks ago to explain my level!

    for me all this is new and I try to read and learn.
    I update my 6450, active the poe for the AP successfully all run.

    Next step :
    As I have an ICX-6450 with the firmware R I want to do the routing by the ICX.
    if it's a bad idea for a beginner like me tell me :)


    WAN 1000baseT <full-duplex> redacted
    LAN 1000baseT <full-duplex> 192.168.0.1
    VLAN50_WIFI_ENFANT 192.168.50.1
    VLAN_IOT 192.168.60.1
    VLAN40_STEVE 192.168.40.1
    VLAN30_HELENE_AUTRE 192.168.30.1

    I have my pfsense in 1/1/1 with this vlan : 30 - 40 - 50 - 60
    I have a Nano HD wifi in 1/1/2 with this vlan : 30 - 40 - 50 - 60
    1/1/1 is in dual mode
    1/1/2 is in dual mode


    From what I read it takes a dhcp server separate from the switch and my pfsense and do some ve on the switch.
    ---------------------------------------------------
    here's what I'm thinking of doing but I'm scared to get started

    1 install isc-dhcp-server on a raspberry pi 3
    apt install isc-dhcp-server
    nano /etc/default/isc-dhcp-server
    INTERFACESv4="interface of the Pi"

    nano /etc/dhcp/dhcpd.conf
    option domain-name "$yourdomainhere";
    option domain-name-servers 192.168.0.1 ( my pfsense )
    default-lease-time 3600;
    max-lease-time 7200;
    authoritative;

    # exemple VLAN 50 wifi enfant, VE Int 50
    subnet 192.168.50.0 netmask 255.255.255.0 {
    option routers 192.168.50.1;
    option subnet-mask 255.255.255.0;
    range 192.168.50.1 192.168.50.20;
    }

    # exemple VLAN40_STEVE, VE Int 40
    subnet 192.168.40.0 netmask 255.255.255.0 {
    option routers 192.168.40.1;
    option subnet-mask 255.255.255.0;
    range 192.168.40.1 192.168.40.100;


    #exemple static ip
    host plex {
    hardware ethernet xx:xx:xx:xx:xx:xx;
    fixed-address 192.168.40.200;
    option host-name "plex";
    }

    # exemple dhcp for transport pfsense -> ICX
    subnet 192.168.0.0 netmask 255.255.255.248 {
    option routers 192.168.0.1;
    option subnet-mask 255.255.255.248;
    range 192.168.0.1 192.168.0.7;
    }



    2 configure icx6450
    ( what I read "you almost never assign an IP directly to a physical interface in layer 3 firmware/switches"

    #"add a vlan for the route to pfsense port 1/1/1"
    vlan 100
    untagged e 1/1/1
    router-interface ve 100
    exit
    int ve 100
    ip addr 192.168.0.2/29

    #"add ve for vlan 50"
    vlan 50
    router-interface ve 50
    exit
    int ve 50
    ip addr 192.168.50.1

    ip helper-adress n xxx.xxx.xxx.xxx

    - where n is a number starting from 1. If this is the only "helper", use 1.
    - xxx.... is the IP address of the DHCP server.

    3 add a route ( I do not know how to do... I see this link but I don't understand)
    dont forget that pfsense will also need a route to know how to reach all the vlans, see the diagram here: https://forums.servethehome.com/ind...ware-trouble-routing-vlans.21113/#post-196527

    question in pfsense /system/Routing/static route/
    I see destination network
    I see Gateway, find what to use ?

    4 learn ACL
     
    #2483
  4. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    60
    Likes Received:
    28
    Okay, I'm going to lay out a couple of things here that I see that are wrong with your examples. One, your dhcpd.conf examples are configuring to hand out your router IP address. You want the "range" option to be something more like, for example on VLAN50, "range 192.168.50.50 192.168.50.200". If done that way it will issue IPs only from between 50.50 and 50.200 to the VLAN 50 clients.

    Second, you NEVER want to use DHCP to configure IPv4 addresses on a router or switch. Leave out your DHCP for transport section. I would simply configure 192.168.0.1 for the pfSense LAN interface, and 192.168.0.2 for the ve100 interface.

    For your ICX configurations, there's some things you need to do first - specifically, you would need to remove 1/1/1 from the tagged VLANs before it will allow you to add the untagged one:

    Code:
    config t
    interface ethe 1/1/1
    no dual-mode
    !
    vlan 30 40 50 60
    no tagged ethe 1/1/1
    !
    vlan 100
    untagged ethe 1/1/1
    !
    end
    Alternately, since your current setup is working, I would strongly recommend you use a different port for the new configuration - say, 1/1/3, or 1/1/48, just something else. Then configure THAT port for the new setup on VLAN 100. That will mean you can configure it, move the cable to the new port, and if it doesn't work right away you can move back.

    For your routes, you want Destination Network on pfSense to be 192.168.0.0/16 (or 192.168.0.0 netmask 255.255.0.0) and Gateway to be the ve100 interface IP (192.168.0.2). On the ICX, you need to configure "ip route 0.0.0.0/0 192.168.0.1" in order for traffic to get past the switch in the first place. Remember that routing will always use the most specific (smallest IP range) route first, so even though 192.168.0.0/16 covers all of your VLANs, it will prefer the routes for /24 ranges over the larger one. This also means you need to pick a VLAN to test from and disable its interface on the pfSense before you switch over, or it will not work.
     
    #2484
    French Chamallow likes this.
  5. French Chamallow

    French Chamallow New Member

    Joined:
    Mar 15, 2019
    Messages:
    17
    Likes Received:
    3
    Thank you Wolfstar!
    It's much clearer
    I'm going to watch it tonight :) UTC +1 here.
     
    #2485
  6. nerdalertdk

    nerdalertdk Fleet Admiral

    Joined:
    Mar 9, 2017
    Messages:
    134
    Likes Received:
    53
    Hi

    Can you use a breakout cable on this ICX7400-1X40GQ module for the icx7400 serie ?
     
    #2486
  7. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    60
    Likes Received:
    28
    Not so far as I'm aware. You would have to replace it with an ICX7400-4x10G* module instead. (GF for Fiber, GC for copper/10GBaseT.)
     
    #2487
    nerdalertdk and fohdeesha like this.
  8. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    correct AFAIK
     
    #2488
    nerdalertdk likes this.
  9. umass1966

    umass1966 New Member

    Joined:
    Nov 2, 2019
    Messages:
    2
    Likes Received:
    1
    Can anybody help me with this question. i have icx7150-c12p switch and want to setup a 10gbe network like freenas server - switch - computer. want suggestions for network card , optic sfp+ and fiber cable . thanks.
     
    #2489
  10. French Chamallow

    French Chamallow New Member

    Joined:
    Mar 15, 2019
    Messages:
    17
    Likes Received:
    3
    did not read this message unless you want to laugh ....
    I forgot to add the firewall rules in my LANTEST :)


    still sorry to annoy you but I assure you that I look before posting!

    I did some tests tonight and My icx does not ping the ip 192.168.2.1
    my pfsense ping well the ip of ve 100 in 192.168.2.2
    I see a continuous traffic on the LANTEST interface ( 120B)

    I added a transit vlan 100
    I added the port 1/1/24
    gave it to him ip 192.168.2.2

    I went in my pfsense and I added a LANTEST on one of the free physical ports of my pfsense in 192.168.2.1

    I added a gateway in System / Routing / Gateways
    name ICX6450
    LANTEST interface
    Gateway 192.168.2.2


    I specify that I left all my config of origin and thus my main lan works on another physical inerface of my pfsense ( 192.168.0.1)

    SSH @ ICX6450 (config) #show ip route
    Total number of IP routes: 3
    Type Codes - B: BGP D: Connected O: OSPF R: RIP S: Static; Cost - Dist / Metric
    BGP Codes - i: iBGP e: eBGP
    OSPF Codes - i: Inter Area 1: External Type 1 2: External Type 2
    Destination Gateway Port Cost Type Uptime
    1 0.0.0.0/0 192.168.0.1 ve 1 1/1 S 4d3h
    2 192.168.0.0/24 DIRECT ve 1 0/0 D 4d3h
    3 192.168.2.0/24 DIRECT ve 100 0/0 D 1h14m


    information on my 1/1/24
    GigabitEthernet1/1/24 is up, line protocol is up
    Port up for 1 hour(s) 38 minute(s) 28 second(s)
    Hardware is GigabitEthernet, address is redacted (bia redacted)
    Configured speed auto, actual 1Gbit, configured duplex fdx, actual fdx
    Configured mdi mode AUTO, actual MDIX
    Member of L2 VLAN ID 100, port is untagged, port state is FORWARDING
    BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
    Link Error Dampening is Disabled
    STP configured to ON, priority is level0, mac-learning is enabled
    Flow Control is config enabled, oper enabled, negotiation disabled
    Mirror disabled, Monitor disabled
    Mac-notification is disabled
    Not member of any active trunks
    Not member of any configured trunks
    No port name
    Inter-Packet Gap (IPG) is 96 bit times
    MTU 1500 bytes, encapsulation ethernet
    300 second input rate: 1000 bits/sec, 1 packets/sec, 0.00% utilization
    300 second output rate: 1008 bits/sec, 1 packets/sec, 0.00% utilization
    7470 packets input, 478194 bytes, 0 no buffer
    Received 1 broadcasts, 0 multicasts, 7469 unicasts
    0 input errors, 0 CRC, 0 frame, 0 ignored
    0 runts, 0 giants
    7489 packets output, 479410 bytes, 0 underruns
    Transmitted 22 broadcasts, 0 multicasts, 7467 unicasts
    0 output errors, 0 collisions
    Relay Agent Information option: Disabled


    my ping

    SSH@ICX6450#ping 192.168.0.1
    Sending 1, 16-byte ICMP Echo to 192.168.0.1, timeout 5000 msec, TTL 64
    Type Control-c to abort
    Reply from 192.168.0.1 : bytes=16 time=1ms TTL=64
    Success rate is 100 percent (1/1), round-trip min/avg/max=1/1/1 ms.
    SSH@ICX6450#ping 192.168.2.1
    Sending 1, 16-byte ICMP Echo to 192.168.2.1, timeout 5000 msec, TTL 64
    Type Control-c to abort
    Request timed out.
    No reply from remote host.
     
    #2490
    Last edited: Nov 6, 2019
  11. microserf

    microserf New Member

    Joined:
    May 20, 2019
    Messages:
    5
    Likes Received:
    2
    So I picked up an ICX 7150-C12P and updated the firmware. On boot, I'm seeing this:
    Code:
    Ruckus Wireless Bootloader: 10.1.14T225 (Nov 15 2018 - 04:59:18 -0800)
    Booted from partition 2
    DRAM:  Validate Shmoo parameters stored in flash ..... OK
    ICX7150-12 (POE), PVT1
    SYS CPLD VER: 0x4 Released Ver: 0xa
    device 0 offset 0x0, size 0xc0000
    Enter 'b' to stop at boot monitor:  0
    device 0 offset 0x0, size 0xc0000
    bootdelay: ===
    Booting image from Primary
    NAND read: device 0 offset 0x0, size 0x2000000
    Skipping bad block 0x00a00000
     33554432 bytes read: OK
    
    The second last line, "Skipping bad block 0x00a00000", concerned me so I flashed the firmware again but there was no change. The switch appears to boot fine. Comments or suggestions?
     
    #2491
  12. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    I would reflash the bootloader as well (it's on the same SPI flash chip) but it looks like it's detected the block and marked it off which is pretty normal, you should be fine. When you need to worry is when the number of bad blocks starts rising over time, then your flash is failing
     
    #2492
    microserf likes this.
  13. microserf

    microserf New Member

    Joined:
    May 20, 2019
    Messages:
    5
    Likes Received:
    2
    But it's all shiny and new :(. I re-flashed the bootloader but no change. What's the mailing address for Fohdeesha's SMD Solderworks?
     
    #2493
  14. aflow

    aflow New Member

    Joined:
    Apr 13, 2016
    Messages:
    2
    Likes Received:
    0
    Hi,
    Is it possible to made this modification with the ICX6450?.

    Thanks.

     
    #2494
  15. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    That modification is for as QSFP breakout cable and the ICX6450 has no QSFP ports, so no :) If you want to rewrite an SFP+ optic, 99% of them have the eeprom locked so you won't be able to write anything. The ICX6450 runs linux so it does not have the same commands, the only I2C write commands I remember are "dm i2c-write" but it only takes one byte at a time so you'll have to run that ~100 times and you'll need to find out the I2C ID of the optic as well. Don't have one powered on at the moment to test
     
    #2495
  16. am45931472

    am45931472 Member

    Joined:
    Feb 26, 2019
    Messages:
    43
    Likes Received:
    8
    Is it possible to manually change the fan speed on the Brocade Ruckus 7150-24p? the ruckus website seems to say that fans can only be automatically controlled. I've tried fan-speed via CLI and it is not recognized as a command. I've swapped out the stock fans for some noctua ones going from an audible but quiet switch to a basically silent switch. Temps are good-ok, between 65-75c, but I think even at full speed these noctua fans would be near silent. 72c seems to be the temp threshold for high speed.
     
    #2496
  17. microserf

    microserf New Member

    Joined:
    May 20, 2019
    Messages:
    5
    Likes Received:
    2
    Code:
    ICX7150-Boot>nand info
    
    Device 0: nand0, sector size 1024 KiB, Micron NAND 2GiB
      Page size       4096 b
      OOB size         224 b
      Erase size   1048576 b
      subpagesize     4096 b
      options     0x   10200
      bbt options 0x       0
    ICX7150-Boot>nand bad
    
    Device 0 bad blocks:
      00a00000
      05a00000
      05b00000
    
    Gah.
     
    #2497
  18. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117

    you'll be aight
     
    #2498
  19. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    nope. if it gets too hot it'll ramp the fans up no need to worry about it
     
    #2499
  20. levi danzer

    levi danzer New Member

    Joined:
    Sep 29, 2019
    Messages:
    3
    Likes Received:
    1
    Just a heads up to any one looking. There is a icx6650-56-e-adv on ebay right now. The seller countered with $300.

    Debated on picking it up... but I grabbed an sx6036 for my 40gbe needs...

    -Levi
     
    #2500
    fohdeesha likes this.
Similar Threads: Brocade Series
Forum Title Date
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019
Networking Brocade VDX 6720 - what do I need to know? Sep 16, 2019

Share This Page