Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Serhan

Member
Sep 22, 2017
89
27
18
I am trying to stack two 7450's that are 70 feet apart. What is the cheapest way of doing this? Any suggestions?
 

Serhan

Member
Sep 22, 2017
89
27
18
Can you run new cables between them, or do you have existing cables that you must use?
I have 5 runs of OM4 LC/LC fiber between these two locations. Those were for 5 servers connected to one of the 7450s which was the previous set-up. I had. I wish there could be a way to repurpose those pre-terminated lc/lc cables to stack the 7450s
 

Ruchira

New Member
Mar 11, 2022
1
0
1
I bought an ICX 7250-24P too :D I unplugged one fan out of two but It is significantly louder than my dell R630 at 20% fan load. I'm going to replace the fans and I'm torn between these 2 options




Can anyone help to select the best fit between these?

Thanks
 

LodeRunner

Active Member
Apr 27, 2019
540
227
43
I have a little Question to the License. I have bought an Brocade VDX6740-24.

I don´t know the exactly meaning of the output of "show license" and "show dpod".

Here is the output.

<snipped console output>

Does this mean, from 24 Ports are 16 for 10Gbit and also all 4 QSFTP Ports are licensed for 40 Gbit, yes?

Or does this mean 16 Ports @ 10 Gbit and 24 Ports @ 1Gbit too?

Why does a VDX6740-24 with normaly 24 Ports show 48 / 40 Ports? Can i license this 24 Port one to full 48 Ports? The Hardware is in the switch?

Thanks for your help :) I'am new at Brocade - sorry for the i think stupid question :D
As I understand the output you provided, the switch natively comes with 24 of the ports licensed for 10G operation and the remaining 24 require POD licensing. So 24 port base license, plus 16 port POD license means you have 40 ports that can be brought online in at 10G, plus the 4x40 G port license. It looks like as ports are brought up, the licenses are first assigned from the base pool; once that is consumed, it will start issuing licenses from the POD pool.

To get all 48 ports active at 10G you'd need to find an additional 8 port POD license.

There was a dedicated thread: Brocade VDX 6740 | ServeTheHome Forums Posters in that thread might have additional help/guidance for you in terms of firmware and licensing issues and how they might be resolved.
 

LodeRunner

Active Member
Apr 27, 2019
540
227
43
Hello LR, trying to understand something. I put the static route in my laptop and can now ping devices on VLAN 2, VLAN 3 and those VLANs can ping VLAN 1. Also, VLAN 2 and VLAN 3 can ping between themselves, however, they could do this before the static route was added.

What I am trying to figure out is why the static route would go on the source device and not the device doing the routing, i.e. Nighthawk or the ICX 6450?
The static route on your laptop is to tell it how to get to the other subnets. Since those subnets are not connected via your Nighthawk, and your Nighthawk is your default gateway, you have to tell your client (source) how it can reach those networks. For devices in VLAN 2 and 3, they're using the ICX as the gateway. The ICX knows what networks are directly connected and can route them; then anything that does not match a direct connected network is sent to the ICX's default gateway, the Nighthawk.

In your specific case, this is required because the Nighthawk and the ICX are in the same subnet.

If your Nighthawk had the ability to do static routes, the more enterprise correct way of setting this up would be
  • each VLAN has its own subnet (you have this)
  • each VLAN has a VE with IP
  • a dedicated VLAN between the ICX and firewall; this VLAN has its own subnet
  • ICX uses firewall VLAN IP as its upstream gateway
  • The firewall has reverse routes for each subnet using the ICX IP as its gateway
Theoretically, you could try setting the gateway on your laptop to use the ICX IP and the ICX would send any non-local traffic to the Nighthawk, but the Nighthawk can directly see your client, so return traffic would come direct instead of going back through the ICX leading to asymmetric routing which generally causes issues.
 

Chow

New Member
Mar 15, 2022
11
0
1
As I understand the output you provided, the switch natively comes with 24 of the ports licensed for 10G operation and the remaining 24 require POD licensing. So 24 port base license, plus 16 port POD license means you have 40 ports that can be brought online in at 10G, plus the 4x40 G port license. It looks like as ports are brought up, the licenses are first assigned from the base pool; once that is consumed, it will start issuing licenses from the POD pool.

To get all 48 ports active at 10G you'd need to find an additional 8 port POD license.

There was a dedicated thread: Brocade VDX 6740 | ServeTheHome Forums Posters in that thread might have additional help/guidance for you in terms of firmware and licensing issues and how they might be resolved.
Thank´s a lot LodeRunner ;-) I will write there ;-)

Ah ok that´s very cool. I thought, that i have only 16 Ports @ 10 Gbit and 4 Ports on 40 Gbit :-D

But you think a VDX6740-24 Model comes with normaly 24 x 10 Gbit Ports in Standard, yes? And the Licenses goes up on top, right?
 

hibby50

New Member
Mar 17, 2022
3
0
1
Hey Guys, I bought a 6610 based on this thread and I could use some help. I'm having an issue configuring extended acls. I have a port tagged with multiple vlans (10,20,40,50) going to an access point, and a machine with an untagged LAG on vlan 50. The problem is on my inbound rule (mgmt-in) for vlan 50 I have `permit tcp any any established` And when that rule is in the inbound acl, the other vlans can ssh into the device on the LAG. If I remove it they can't but then it also cant get out :)

I also tried enable acl-per-port-per-vlan despite not knowing what it means, but that did nothing

Edit: I dumbed it down and put one machine on an untagged vlan 20 port and the other on the LAG untagged vlan 50 port with only 1 acl on the vlan 50 ve: 'permit tcp any any established' and I was able to initiate an ssh session from the vlan 20 machine. Totally stumped.


Code:
SSH@ICX6610-48P-Router(config)#show lag

=== LAG "hibsrv-LAG" ID 2028 (dynamic Deployed) ===

LAG Configuration:
Ports: e 1/3/1 to 1/3/2
Port Count: 2
Primary Port: 1/3/1
Trunk Type: hash-based
LACP Key: 22028
LACP Timeout: long
Deployment: HW Trunk ID 2
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/3/1 Up Forward Full 10G 2028 No 50 0 748e.f8ff.fc4e
1/3/2 Up Forward Full 10G 2028 No 50 0 748e.f8ff.fc4e

Port [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/3/1 1 1 22028 Yes L Agg Syn Col Dis No No Ope
1/3/2 1 1 22028 Yes L Agg Syn Col Dis No No Ope


Partner Info and PDU Statistics
Port Partner Partner LACP LACP
System ID Key Rx Count Tx Count
1/3/1 65535-246e.9600.5080 15 11 9
1/3/2 65535-246e.9600.5080 15 8 8
Code:
SSH@ICX6610-48P-Router(config)#sh run int e 1/1/13

interface ethernet 1/1/13
inline power
!
Code:
SSH@ICX6610-48P-Router(config)#sh vlan br     

PORT-VLAN 10, Name guest, Priority level0, Spanning tree Off

Untagged Ports: None
Tagged Ports: (U1/M1) 13 14 15
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: Disabled

PORT-VLAN 20, Name iot, Priority level0, Spanning tree Off
Untagged Ports: (U1/M1) 39
Tagged Ports: (U1/M1) 13 14 15
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: Disabled

!

PORT-VLAN 50, Name mgmt, Priority level0, Spanning tree Off

Untagged Ports: (U1/M1)  37  38
Untagged Ports: (U1/M2) 1 2 3 4 5 6 7 8 9 10
Untagged Ports: (U1/M3) 1 2 3 4 5 6 7 8
Tagged Ports: (U1/M1) 13 14
Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
Monitoring: Disabled
Code:
Extended IP access list  mgmt-in : 10 entry

ACL Remark:  ALLOW DHCP
permit udp any any eq bootps
permit udp any any eq bootpc
ACL Remark: ALLOW DNS
permit tcp 192.168.50.0 0.0.0.255 host 10.0.0.2 eq dns
permit udp 192.168.50.0 0.0.0.255 host 10.0.0.2 eq dns
ACL Remark: ALLOW ESTABLISHED TCP TRAFFIC
permit tcp any any established
ACL Remark: ALLOW mDNS
permit udp any host 224.0.0.251 eq 5353
ACL Remark: ALLOW HERE TO ANYWHERE
permit ip 192.168.50.0 0.0.0.255 any
ACL Remark: DENY INTER-VLAN TRAFFIC
deny ip 192.168.0.0 0.0.255.255 any
deny ip 10.0.0.0 0.255.255.255 any
deny ip 172.16.0.0 0.15.255.255 any
 
Last edited:

hibby50

New Member
Mar 17, 2022
3
0
1
I just solved it. I realized I had a fundamental misunderstanding of inbound and outbound acl. It is inbound and outbound to the PORT, not inbound outbound to the vlan/subnet :eek:
 

LodeRunner

Active Member
Apr 27, 2019
540
227
43
Thank´s a lot LodeRunner ;-) I will write there ;-)

Ah ok that´s very cool. I thought, that i have only 16 Ports @ 10 Gbit and 4 Ports on 40 Gbit :-D

But you think a VDX6740-24 Model comes with normaly 24 x 10 Gbit Ports in Standard, yes? And the Licenses goes up on top, right?
I'm taking what the output you provided says at face value:
Code:
40 10G port assignments are provisioned for use in this switch:
        24 10G port assignments are provisioned by the base switch license
        16 10G port assignments are provisioned by the 10G Port Upgrade license
14 10G ports are assigned to installed licenses:
        14 10G ports are assigned to the base switch license
         0 10G ports are assigned to the 10G Port Upgrade license
 

Chow

New Member
Mar 15, 2022
11
0
1
Thanks for your answer LodeRunner ;-)

Hmm ok, i thought that the "base switch license" is only 1 Gbit, not 10 Gbit?!
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,729
3,080
113
33
fohdeesha.com
I have 5 runs of OM4 LC/LC fiber between these two locations. Those were for 5 servers connected to one of the 7450s which was the previous set-up. I had. I wish there could be a way to repurpose those pre-terminated lc/lc cables to stack the 7450s
the 7450's happily stack over 10gbe as well, if you have the 4x 10gbe port modules for your 7450s. if you only have 40gb qsfp modules, there's the much cheaper Kaiam LR4 lite modules that would allow you to use your existing lc fiber if it were singlemode, but you say it's multimode - there's multimode SR4 BiDi transceivers like @kpfleming linked, but I don't know if the ICX7450 QSFP+ ports can supply enough power for SR4 BiDi (haven't seen anyone try, although the module he links claims only 1.5W of draw)
 
  • Like
Reactions: Serhan

RoachedCoach

Member
Feb 4, 2020
35
41
18
I bought an ICX 7250-24P too :D I unplugged one fan out of two but It is significantly louder than my dell R630 at 20% fan load. I'm going to replace the fans and I'm torn between these 2 options




Can anyone help to select the best fit between these?

Thanks
I replaced mine with Mechatronix, they've been perfect for me.

Best bet is sticking a fan on the ASIC as well, makes all the difference.
 
  • Like
Reactions: Ruchira

kache

New Member
Jun 27, 2020
15
2
3
Hi guys,

Finally installed the icx7250 for my secondary network, replacing the messy server-to-server connection routed by an pfsense VM, but I'm wondering concerning the power consumption.
This is the icx7250-24, so only 24 ports and no POE, yet the power consumption of the whole rack went up from ~480w up to ~540w, an increase of about 60w, and this is with 4 ports used (3x 10G, 1x 1G)

Does that seem correct? The ICX6450 48P with 2 POE devices connected takes around ~100W, so 60w for the icx7250-24 without POE seems a bit excessive.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,729
3,080
113
33
fohdeesha.com
Hi guys,

Finally installed the icx7250 for my secondary network, replacing the messy server-to-server connection routed by an pfsense VM, but I'm wondering concerning the power consumption.
This is the icx7250-24, so only 24 ports and no POE, yet the power consumption of the whole rack went up from ~480w up to ~540w, an increase of about 60w, and this is with 4 ports used (3x 10G, 1x 1G)

Does that seem correct? The ICX6450 48P with 2 POE devices connected takes around ~100W, so 60w for the icx7250-24 without POE seems a bit excessive.
what power monitoring device are you using? it sounds grossly inaccurate if it measured any 6450 variant at a hundred watts
 

kache

New Member
Jun 27, 2020
15
2
3
what power monitoring device are you using? it sounds grossly inaccurate if it measured any 6450 variant at a hundred watts
Hi,

Not a bad point, currently I'm taking the total power consumption at the UPS, minus the two servers (data from IDRAC), minus an estimated 100w for the rest of the infra (ISP modem, 2x lenovo tiny, 2x laptops, 1x shuttle small PC, multiple rack fans).

I might need to get my hands on some IOT devices to analyze power consumption for each device individually to have a clearer idea of the power consumption.
 

zunder1990

Active Member
Nov 15, 2012
209
71
28
I am having problems getting the license to apply, I got a stack of two 6610. I use the switches for the Linux fest that I run.

Code:
  Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
    UNIT 1: compiled on Apr 23 2020 at 12:11:06 labeled as FCXS08030u
                (7723621 bytes) from Primary FCXS08030u.bin
        SW: Version 08.0.30uT7f1
    UNIT 2: compiled on Apr 23 2020 at 12:11:06 labeled as FCXS08030u
                (7723621 bytes) from Primary FCXS08030u.bin
        SW: Version 08.0.30uT7f1
  Boot-Monitor Image size = 370695, Version:10.1.00T7f5 (grz10100)
  HW: Stackable ICX6610-24F
==========================================================================
UNIT 1: SL 1: ICX6610-24F 24-port Management Module
         Serial  #: 2ax5o2jk68e
         License: BASE_SOFT_PACKAGE   (LID: H4CKTH3PLN8)
         P-ENGINE  0: type E02B, rev 01
==========================================================================
UNIT 1: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 1: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
UNIT 2: SL 1: ICX6610-48P POE 48-port Management Module
         Serial  #: BXK3845L00A
         License: BASE_SOFT_PACKAGE   (LID: dzmINJKnFFc)
         P-ENGINE  0: type E023, rev 01
         P-ENGINE  1: type E023, rev 01
==========================================================================
UNIT 2: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 2: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
  800 MHz Power PC processor 8544E (version 0021/0023) 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 30 minute(s) 54 second(s)
STACKID 2  system uptime is 31 minute(s) 3 second(s)
The system started at 06:00:23 GMT+00 Thu Feb 07 2036

 The system : started=warm start         reloaded=by "reload"
My stack unit ID = 1, bootup role = active
Code:
nocsw.self.lan#show stack
T=33m23.4: alone: standalone, D: dynamic cfg, S: static
ID   Type          Role    Mac Address    Pri State   Comment
1  S ICX6610-24F   active  cc4e.24c2.3614   0 local   Ready
2  S ICX6610-48P   standby cc4e.24f8.83b4   0 remote  Ready

    active       standby
     +---+        +---+
 =2/6| 1 |2/1==2/6| 2 |2/1=
 |   +---+        +---+   |
 |                        |
 |------------------------|
Standby u2 - protocols ready, can failover or manually switch over
Current stack management MAC is cc4e.24c2.3614
Code:
nocsw.self.lan#show license
Index    Lic Mode        Lic Name               Lid/Serial No  Lic Type    Status     Lic Period    Lic Capacity
Stack unit 1:
4        Node Lock       ICX6610-10G-LIC-POD    H4CKTH3PLN8    Normal      Active     Unlimited         8
5        Node Lock       ICX6610-ADV-LIC-SW     H4CKTH3PLN8    Normal      Active     Unlimited         1
6        Node Lock       ICX-MACSEC-LIC         H4CKTH3PLN8    Normal      Active     Unlimited         1
Stack unit 2:
1        Node Lock       ICX6610-10G-LIC-POD    H4CKTH3PLN8    Normal      Invalid    Unlimited         8
2        Node Lock       ICX6610-ADV-LIC-SW     H4CKTH3PLN8    Normal      Invalid    Unlimited         1
3        Node Lock       ICX-MACSEC-LIC         H4CKTH3PLN8    Normal      Invalid    Unlimited         1
nocsw.self.lan#
Code:
nocsw.self.lan#copy tftp license 10.1.0.1 1-6610-ports.xml unit 2
nocsw.self.lan#Flash Memory Write (8192 bytes per dot) .
Copy Software License from TFTP to Flash Done.

Copy software license to stack unit 2 success
Download request from active unit 1 mac = cc4e.24c2.3614
Downloading - $$$license
Done.
Can't add the license string - 93 (DUPLICATE_LICENSE)