Connect those two servers together directly with a qsfp dac and try again, you're probably CPU/interrupt bound somewhere, could be marginal cabling/optics as well, check port statistics on every port in the path looking for discards/crc errors etc
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
ok so directly attached them and am getting 12.6Gbps. Swapped cable out for a different one and didn't make any difference. I'm wondering if these cards need different firmware or modes or something. Forgive me it's the first time I've dealt with any 40Gb nics. the specific cards that I bought off ebay were Mellanox MCX4131A-GCAT_C05 ConnectX-4 LX 50GbE PCIe Network Card Newest Firmware | eBay .
lshw -C net output:
What do you think? its not a cpu issue. when I do 2 VMs on the same ESXI server, and have SR-IOV configured and have them in hte same port group on the same vswitch and using VMXNET3 , I do get over 35Gbps, but in that scenario isn't it getting short circuited by the VMXNET3 driver and vSwitch and not actually hitting the physical nic and switch, right? I would think that rules out a few bottlenecks though.
Last edited:
any chance it's these cables? I tried 3 different ones. Are these junk? Do I need official Mellanox cables like MCP1700-B003E or Mc2210128-003 ?
On my 7250, I see the following ip options for multicast:
Also, do I need to enable both multicast routing and snooping to get things like sonos and apple bonjour to work?
- ip multicast
- ip multicast-routing
- ip multicast-nonstop-routing
Also, do I need to enable both multicast routing and snooping to get things like sonos and apple bonjour to work?
- ip multicast version 3
- router pim
Last edited:
I'm still waiting for the $10 40G-LR4 Lite (uses duplex LC SMF) transceivers to arrive, but I just noticed that the ICX6XXX series administrate guide lists the Brocade optics that will work with the ICX6610 (p. 229). It specifically lists the 57-1000263-01 which is a 40G-LR4 transceiver with a 10km reach using duplex SMF fiber. So, it's a pretty good bet the generic modules will work as well. While the datasheet indicates that the SFP+ ports are limited to DAC, 10G-SR, and 10G-LR, the administrative manual also lists Brocade 10G-ER (40km) and 10G-ZR (80km), but it says the ZR will only run on the 1/3/8 port. I doubt anyone would need 40km or 80km optics, but it's nice to know the switch will accept almost anything. I assume BiDi optics will work fine as well.
As for the SSH issue, with only
I see the following using putty:
After adding
I get the expected result:
As for the SSH issue, with only
Code:
ip ssh password-authentication no
Code:
login as: jason
Keyboard-interactive authentication prompts from server:
| Password:
End of keyboard-interactive prompts from server
SSH@ICX6610-48p>
Code:
ip ssh interactive-authentication no
Code:
No supported authentication methods available (server sent: publickey)
Last edited:
Read up in the manual (first link below). You can also search for those commands as well (use the second link below) and compare the difference. I had to turn igmp snooping and such off to get Homekit to work over my icx7150.
RUCKUS FastIron IP Multicast Configuration Guide, 08.0.95
RUCKUS FastIron Command Reference Guide, 08.0.95
FYI, for anyone still on the 9000 series firmware, 9010 was released yesterday.
RUCKUS ICX FastIron 09.0.10 (GA) Software Release (.zip)
FastIron 09.0.10
RUCKUS ICX FastIron 09.0.10 (GA) Software Release (.zip)
FastIron 09.0.10
@Vesalius Thx for those links. I have read through them a few times but still not exactly sure what do. Did you turn igmp snooping off or on? Isn't it off by default? I did the following but still don't have connectivity for my sonos: I am sure I have it completely wrong.
Code:
# from main conf t:
ip multicast version 3
# then for each vlan
vlan 110 ( home devices with sonos controller app)
multicast version 3
vlan 130 ( iot vlan with sonos devices)
multicast version 3
Last edited:
I don't have sonos so can't help there. Do not remember what the default is now, but I do know when/if igmp snooping was on for my network/vlan homekit devices went unresponsive. I had to make sure igmp snooping was off. This was even when my homekit hubs and devices lived on the same vlan.
If you are trying to cross vlans for sonos control and or any multicast control then you will need something to do that for you. It gets complicated quickly, but what are you using to do the routing for your network? Things like Avahi among others come into play.
Will a 10G-SR or 10G-LR transceiver enable its laser if there is no link on an ICX6610? The Brocade docs indicate that there is a command (port-down-disable-laser) for the ICX7250 and ICX7450, but the docs state this isn't supported on the ICX6XXX switches (Commscope Technical Content Portal). I see digital optical monitoring output for the active ports but not for the inactive ports, which would suggest the laser is not on. The show media and show media validation commands do show the inactive transceivers. I removed the transceivers and they aren't warm, so there's no indication the laser is on. Just wondering if I should remove them. I plan to do two runs to each room for future needs and was planning to just keep it connected, as I would do with a copper port.
Code:
Port Supported Vendor Type
----------------------------------------------------------------------
1/3/1 Yes BROCADE Type : 10GE SR 300m (SFP +)
1/3/2 Yes BROCADE Type : 10GE SR 300m (SFP +)
1/3/3 Yes BROCADE Type : 10GE SR 300m (SFP +)
1/3/4 Yes BROCADE Type : 10GE SR 300m (SFP +)
1/3/5 Yes BROCADE Type : 10GE LR 10km (SFP +)
SSH@ICX6610-48p(config)#show optic 1/3/1
Port Temperature Tx Power Rx Power Tx Bias Current
+----+-----------+--------------+--------------+---------------+
1/3/1 32.0195 C -002.1666 dBm -002.1197 dBm 5.934 mA
Normal Normal Normal Normal
SSH@ICX6610-48p(config)#show optic 1/3/2
Port Temperature Tx Power Rx Power Tx Bias Current
+----+-----------+--------------+--------------+---------------+
1/3/2 32.7656 C -002.5305 dBm -002.7140 dBm 5.796 mA
Normal Normal Normal Normal
SSH@ICX6610-48p(config)#show optic 1/3/3
Port Temperature Tx Power Rx Power Tx Bias Current
+----+-----------+--------------+--------------+---------------+
1/3/3 34.4648 C -002.2380 dBm -002.7794 dBm 5.492 mA
Normal Normal Normal Normal
SSH@ICX6610-48p(config)#show optic 1/3/4
SSH@ICX6610-48p(config)#show optic 1/3/5I believe that you'll need Avahi to allow network discovery across VLANs for Homebridge. I set that up when I was trying to get Homebridge to work in my segmented network. See Homekit and Avahi.
thanks for verifying, guide updated: disable interactive auth for SSH configs · Fohdeesha/lab-docu@8bf0361
i fixed it with a mdns server, it’s a simple python script that relays the broadcast since mDNS normally have ttl of 1 hop
Not home right now so can’t see the script name
Sorry, I’m using pfsense to route the relevant VLANs. However, I can tell you that HomeBridge required me to enable Avahi on pfsense with reflection enabled (repeats mdns packets across subnets). Pfsense will give you the option to choose the specific subnets where you want to enable this feature. Once I did that, HomeBridge worked immediately with no additional configuration. Since neither the IoT VLAN or the VLANs I use for my Wifi networks require > 1 Gb of traffic, I just have pfsense handle routing.
For greater security, you can filter for only the services you wish to reflect. On Linux,
Code:
avahi-browse —all
Last edited:
I bought a 6450 off eBay. In the process of getting it configured but the older boot software is bit troublesome.
Anyway, figured out this switch is about 7 years old. Is there any significant hardware upgrades in the past 7 years or so? Wondering how this switch compares to the newer models. Couldn't find any information online about the old versions.
Anyway, figured out this switch is about 7 years old. Is there any significant hardware upgrades in the past 7 years or so? Wondering how this switch compares to the newer models. Couldn't find any information online about the old versions.
What trouble are you having specifically? Did you follow the guide to upload the bootloader and OS firmware to the latest version? If so, you can check the 68-page product support matrix to determine exactly which features are supported. This manual can be found in the ICX6xxx Manuals folder.
Any router peeps out there willing to lend a hand? I am configuring OSPF and having an issue with passive interfaces. I come from the Cisco world where I would passive default the config then no passive the links I want to form neighbors on. I see the passive default command in the Brocade but I cant figure out how to no passive the interfaces I want. Transit Vlan specifically.
I did read the manual but it only reverences the passive-interface-default command and not how to enable an interface.
I did read the manual but it only reverences the passive-interface-default command and not how to enable an interface.
if you follow the guide linked in the OP, it should get you updated with pretty simple steps regardless of how old the existing bootloader is
as for "new" features, not really outside of new connectivity options like multigig/2.5gbE etc
So, those $13 Bidi 40gbE transceivers work with zero issues on the 6610. Also, he's accepting best offers of $8 - absolute steal. 40gb over regular cheap duplex singlemode fiber, thanks to @jasonwc for the find. auction - XQX2502 KAIAM QSFP+40G-LR4 Lite OPTICAL MODULE NEW PULLS | eBay (if link dies, search around for KAIAM XQX2502)
Tested in all 4 ports as I recall talk of one port being higher power for ZR factory optics, and these work in all 4 slots including the 4x10gbE slots:
Code:
ICX6610-24P Router#show stack con
Probing the topology. Please wait ...
ICX6610-24P Router#
standby active
+---+ +---+
2/6| 2 |2/1==2/1| 1 |2/6
+---+ +---+
trunk probe results: 1 links
Link 1: u1 -- u2, num=1
1: 1/2/1 (T0) <---> 2/2/1 (T0)
Code:
ICX6610-24P Router#show media e 1/2/1
Port 1/2/1:Type : 40G QSFP Module
Vendor Name: KAIAM CORP Serial Num: KD60630129 Revision: 1A
ICX6610-24P Router#show media e 2/2/1
Port 2/2/1: Type : 40G QSFP Module
Vendor: KAIAM CORP Version: 1A
Part# : Serial#: KD60628356
Code:
ICX6610-24P Router#ICX6610-24P Router#show int e 1/2/1
40GigabitEthernet1/2/1 is up, line protocol is up
Port up for 10 minute(s) 31 second(s)
Hardware is 40GigabitEthernet, address is cc4e.243d.3eff (bia cc4e.243d.3eff)
Interface type is 40Gig Fiber
Configured speed 40Gbit, actual 40Gbit, configured duplex fdx, actual fdx
Code:
ICX6610-24P Router#show stack con
Probing the topology. Please wait ...
ICX6610-24P Router#
active
+---+ +---+
2/6| 2 |2/1==2/1| 1 |2/6
+---+ +---+
trunk probe results: 1 links
Link 1: u1 -- u2, num=4
1: 1/2/2 (T0) <---> 2/2/2 (T0)
2: 1/2/3 (T0) <---> 2/2/3 (T0)
3: 1/2/4 (T0) <---> 2/2/4 (T0)
4: 1/2/5 (T0) <---> 2/2/5 (T0)
CPU to CPU packets are fine between 2 units.
Last edited: