Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

fohdeesha

Kaini Industries
Nov 20, 2016
2,491
2,674
113
31
fohdeesha.com
Actually was looking for another switch because the 7150-c12p I got after reading this thread is full and I don't have the heart or patience to mix my mikrotik into this mess. What 7250s are they, what are you looking to get for them, and roughly what side of town - I may take one of those off your hands tonight.

I'm looking for L3, as much 10G mixed in as possible, and quiet enough to be drowned out by an air purifier



edit. fix some auto correct
Some poe some not poe, I think they're all the 48 port models but I might be wrong. As for price, I dunno whatever they're going for on ebay minus 20% or something? I won't be around tonight but can make some other time this week work. North side 96th and keystone
 

bcook

New Member
Sep 3, 2021
2
1
3
Does anyone near Indianapolis want to buy any switches? I have so many I need to offload but I don't have the time or energy to deal with shipping. ICX6610s, ICX7250s, ICX6450s, Dell S4810P, Brocade ADX, brocade CES, probably more I'm forgetting
Cool, a fellow Hoosier!

If I wasn't waiting on delivery of an ICX6450-48P, I might make the drive from Evansville to Indy. Funny enough, I've been second-guessing my purchase and wondering if I should have gone with an ICX7250 with PoE. Other than the 4 extra SFP+ ports, would there be a compelling reason to go with the 7250 over the 6450?
 
  • Like
Reactions: fohdeesha

infoMatt

Active Member
Apr 16, 2019
222
99
28
would there be a compelling reason to go with the 7250 over the 6450?
Why not keeping both? :p

7000-series should be better particularly on the software side, and pretty much mandatory if you want to be seriously in business with IPv6 (IIRC, no SLAAC on the 6450 and 6610... or some other things IPv6-related).
 

Shaun Bosworth

New Member
Dec 23, 2017
1
0
1
43
Hey Guys

Long time lurker, first time poster.

Trying to get the 2 rear 40G QSFP stacking ports on my ICX6610-48P to play nice with my Intel XL710-QDA2 card in a Dell R720XD. I have two 3m QSFP DAC cables between the card and the ports on the switch, and the link is negotiated at 40G full duplex on Windows Server, but there is no traffic been allowed, not even DHCP calls for IP address, comes back as unidentified network after a few minutes trying. The switch also shows full 40G on the web interface, as well as the SSH console.

I have disabled the stack configuration and all ports are added to the default VLAN 1 with all ports as untagged. Running the latest firmware from Brocade. Running config below:

Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack disable
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
enable aaa console
hostname brocadesw-rack
ip dhcp-client disable
ip dns domain-list xxxxxxxxx
ip dns server-address 192.168.10.15 192.168.10.16
ip route 0.0.0.0/0 192.168.10.1
!
no telnet server
username root password .....
!
!
clock timezone gmt GMT+02
!
!
ntp
disable serve
server 192.168.10.1
!
!
web-management page-menu
!
!
!
!
!
!
!
interface ethernet 1/1/45
inline power priority 1 power-by-class 4
!
interface ethernet 1/1/46
inline power priority 1 power-by-class 4
!
interface ethernet 1/1/47
inline power priority 1 power-by-class 4
!
interface ethernet 1/1/48
inline power priority 1 power-by-class 4
!
interface ve 1
ip address 192.168.10.3 255.255.255.0
!
!
!
!
!
!
!
!
!
end


Any ideas, as I have checked all the options within the Intel config tool and cannot see what is going on. Any help would be appreciated
 

bcook

New Member
Sep 3, 2021
2
1
3
Why not keeping both? :p

7000-series should be better particularly on the software side, and pretty much mandatory if you want to be seriously in business with IPv6 (IIRC, no SLAAC on the 6450 and 6610... or some other things IPv6-related).
If only :) Don't think I that would fly with the wife, lol, plus my little lab isn't big enough right now to justify two switches like that. If the 6450 is already overkill for my current setup then the 7250 would be even more so....but when is any of what we do with homelabs of any size not overkill? :D

I may still see what fohdeesha has available in the ICX7250-48P or 24P flavor....
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,491
2,674
113
31
fohdeesha.com
Hey Guys

Long time lurker, first time poster.

Trying to get the 2 rear 40G QSFP stacking ports on my ICX6610-48P to play nice with my Intel XL710-QDA2 card in a Dell R720XD. I have two 3m QSFP DAC cables between the card and the ports on the switch, and the link is negotiated at 40G full duplex on Windows Server, but there is no traffic been allowed, not even DHCP calls for IP address, comes back as unidentified network after a few minutes trying. The switch also shows full 40G on the web interface, as well as the SSH console.

I have disabled the stack configuration and all ports are added to the default VLAN 1 with all ports as untagged. Running the latest firmware from Brocade. Running config below:

Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack disable
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
enable aaa console
hostname brocadesw-rack
ip dhcp-client disable
ip dns domain-list xxxxxxxxx
ip dns server-address 192.168.10.15 192.168.10.16
ip route 0.0.0.0/0 192.168.10.1
!
no telnet server
username root password .....
!
!
clock timezone gmt GMT+02
!
!
ntp
disable serve
server 192.168.10.1
!
!
web-management page-menu
!
!
!
!
!
!
!
interface ethernet 1/1/45
inline power priority 1 power-by-class 4
!
interface ethernet 1/1/46
inline power priority 1 power-by-class 4
!
interface ethernet 1/1/47
inline power priority 1 power-by-class 4
!
interface ethernet 1/1/48
inline power priority 1 power-by-class 4
!
interface ve 1
ip address 192.168.10.3 255.255.255.0
!
!
!
!
!
!
!
!
!
end


Any ideas, as I have checked all the options within the Intel config tool and cannot see what is going on. Any help would be appreciated
longshot, but have you tried rebooting the switch?
 

neonclash

New Member
Aug 10, 2015
11
2
3
Does anyone near Indianapolis want to buy any switches? I have so many I need to offload but I don't have the time or energy to deal with shipping. ICX6610s, ICX7250s, ICX6450s, Dell S4810P, Brocade ADX, brocade CES, probably more I'm forgetting
A relative of mine returns to the Indianapolis area this evening, so this is perfect timing. I just can't settle on what hardware would fit my use case the best, so recommendations would be welcome.

The eventual goal for my home setup is reliability and mandatory access control on edge devices to assigned VLANs using 802.1x. Most of the 10gbit (three servers and growing) stays in the rack, so I need a stack there with PoE for my pi hats and backup AP. I also have an office with a PoE AP, and need 10gbit to a workstation there. The Z-series line would be the best fit because of the 2.5Gbase-T that my APs can take advantage of, but they're impossible to find.

Right now I have a 7150-c12p in the office and a 7150-24P in the closet. It's 10gbit from end to end, and I have it working the way I want to with VLANs and even a guest PVLAN. I just need to scale out soon, preferably while adding the redundancy I am after.

The 7250s are still maintained and use less power, but that MACSEC on the 6610 is nice, and I've never played with OpenFlow before, not to mention they're cheaper. What do I lose out on that I might not expect, and what do you recommend?
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,491
2,674
113
31
fohdeesha.com
A relative of mine returns to the Indianapolis area this evening, so this is perfect timing. I just can't settle on what hardware would fit my use case the best, so recommendations would be welcome.

The eventual goal for my home setup is reliability and mandatory access control on edge devices to assigned VLANs using 802.1x. Most of the 10gbit (three servers and growing) stays in the rack, so I need a stack there with PoE for my pi hats and backup AP. I also have an office with a PoE AP, and need 10gbit to a workstation there. The Z-series line would be the best fit because of the 2.5Gbase-T that my APs can take advantage of, but they're impossible to find.

Right now I have a 7150-c12p in the office and a 7150-24P in the closet. It's 10gbit from end to end, and I have it working the way I want to with VLANs and even a guest PVLAN. I just need to scale out soon, preferably while adding the redundancy I am after.

The 7250s are still maintained and use less power, but that MACSEC on the 6610 is nice, and I've never played with OpenFlow before, not to mention they're cheaper. What do I lose out on that I might not expect, and what do you recommend?
I don't think you'd lose out on anything going with the 7250 over the 6610 - MACSEC and openflow don't really have real applications in a setup like that - plus with the 7 series you get newer software/features. Just looked and this is what I have that I'm willing to part with (no PoE 7250s left sadly):

ICX7250-48 X2
ICX7250-24 X1

ICX6610-48P X3
ICX6610-48 X2

ICX6450-24 X2
ICX6450-48P X1

FCX648S X4
FCX648S-HPOE X1

Juniper EX4300 X1
Dell S4810P X1
CES-2024C X1
 

neonclash

New Member
Aug 10, 2015
11
2
3
I don't think you'd lose out on anything going with the 7250 over the 6610 - MACSEC and openflow don't really have real applications in a setup like that - plus with the 7 series you get newer software/features. Just looked and this is what I have that I'm willing to part with (no PoE 7250s left sadly):

ICX7250-48 X2
ICX7250-24 X1

ICX6610-48P X3
ICX6610-48 X2

ICX6450-24 X2
ICX6450-48P X1

FCX648S X4
FCX648S-HPOE X1

Juniper EX4300 X1
Dell S4810P X1
CES-2024C X1
Thanks for the response! I'm gonna poke some ebay listings to see if I can get a couple of 7250-xxPs first. In the meantime, what's your asking price for each 6610-48P?
 
  • Like
Reactions: fohdeesha

NablaSquaredG

Well-Known Member
Aug 17, 2020
604
260
63
Is there any way to *lower* the temperature where the fans spin up on ICX6610?

I'm currently using them in a quite warm environment and the fans spin up every ~30-60s and then spin down again... spin up again... spin down...

I'd like to have them permanently running on level 2...
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,491
2,674
113
31
fohdeesha.com
Thanks for the response! I'm gonna poke some ebay listings to see if I can get a couple of 7250-xxPs first. In the meantime, what's your asking price for each 6610-48P?
cheapest I see on ebay for 48Ps with dual PSUs is 190, I could do 160? Obviously no shipping fee with the local pickup, and they've been graced by the brocade loser himself
 

Rootless

New Member
Aug 9, 2021
7
4
3
I can't wait for the surprise. I love this thread. I now have two ICX-6610, one of the Dell Networking Cable QSFP+ 40GBE - 4X SFP+ 10GBE 3M 27GG5 cables. All of the licenses are good and both units are online. The Dell cable works fine between the two ICX6610, but I'm having problems getting it working with the " Cavium QLogic 1010/1020/1007/1741 10Gbps CNA (rev 01)" installed in my Dell R910.

The card is visible (lspci):
05:00.2 Ethernet controller: Cavium QLogic 1010/1020/1007/1741 10Gbps CNA (rev 01)
05:00.3 Ethernet controller: Cavium QLogic 1010/1020/1007/1741 10Gbps CNA (rev 01)

But the links always show as down both on the server:
6: ens5f2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
link/ether 8c:7c:ff:20:c0:32 brd ff:ff:ff:ff:ff:ff
altname enp5s0f2
7: ens5f3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
link/ether 8c:7c:ff:20:c0:33 brd ff:ff:ff:ff:ff:ff
altname enp5s0f3

and on the switch:
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/7 Down None None None None No 1 0 cc4e.2484.0568
1/2/8 Down None None None None No 1 0 cc4e.2484.0568
1/2/9 Down None None None None No 1 0 cc4e.2484.0568

(the one connected between the switches at the moment is 1/2/10:
SSH@racksw1>sh int br e 1/2/10
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/10 Up Forward Full 10G None No 1 0 cc4e.2484.0568
)

The Dell breakout DAC
SSH@racksw1>sh media e 1/2/7
Port 1/2/7:Type : 40GBASE-Passive Copper
Vendor Name: Amphenol Serial Num: CN027GG554Q4JWR Revision: C

All of the QSFP ports are removed from stacking and stacking is disabled:
SSH@racksw1>sh conf
!
Startup-config data location is flash memory
!
Startup configuration:
!
ver 08.0.30tT7f3
!
stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack disable
stack mac cc4e.2484.0568
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
hostname racksw1
ip dhcp-client disable
ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1
!
no telnet server
username root password .....
no snmp-server
!
!
clock timezone gmt GMT-07
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
!
web-management https
hitless-failover enable
!
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
speed-duplex 10G-full
!
interface ve 1
ip address 192.168.1.58 255.255.255.0
!
!
end

I picked up the Brocade 1020 NIC because I read in a forum post that they just work. I have since found another post that describes them as finicky. Not sure where to go from here. Do I need to pickup something like a MCX354A-QCBT Mellanox CX354A ConnectX-3 VPI QDR IB 10GbE Dual-Port QSFP NIC? If so, why do some of the MCX345A-QCBT list themselves as 10Gb/40Gb, some as 10Gb only, and some as 40Gb only?

Thank you for all of the incredible documentation and information sharing. It is what got me this far. Thank you.
 
  • Like
Reactions: Yunia and fohdeesha