Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Guis are the devils work........
I'm just happy I have things figured out.
Have the 2 10G NC523SFP links up and running as a lag connected to the 6610 with the breakout cable. I dropped 40G optics into the the 2 switches for the uplink. I have the 50M cable, but I still need to switch the polarity.
Not going to bother with a 10G card for the pfsense box, since I am no longer doing vlan control through it, and I only have 200Mbit symetric fiber (FIOS), I don't see the need.
The one issue I had, so far just happened once, and I am not sure if it's related to thermal protection or not.. the links of the 10G card went dead, no carrier. I pulled the module on the switch side and reseated it, no luck, I pulled the transceiver from the cards ont eh VM Server, reseated, no luck, eventually I just rebooted the switch, and they came back. I have not tried rebooting the vmserver as it takes 10 minutes to bootup and start the virtual machines, with some custom scripting because the iscsi mounts are mounted from freenas which is a VM itself (passthrough for 2 m1015 hbas).
I'm just happy I have things figured out.
Have the 2 10G NC523SFP links up and running as a lag connected to the 6610 with the breakout cable. I dropped 40G optics into the the 2 switches for the uplink. I have the 50M cable, but I still need to switch the polarity.
Not going to bother with a 10G card for the pfsense box, since I am no longer doing vlan control through it, and I only have 200Mbit symetric fiber (FIOS), I don't see the need.
The one issue I had, so far just happened once, and I am not sure if it's related to thermal protection or not.. the links of the 10G card went dead, no carrier. I pulled the module on the switch side and reseated it, no luck, I pulled the transceiver from the cards ont eh VM Server, reseated, no luck, eventually I just rebooted the switch, and they came back. I have not tried rebooting the vmserver as it takes 10 minutes to bootup and start the virtual machines, with some custom scripting because the iscsi mounts are mounted from freenas which is a VM itself (passthrough for 2 m1015 hbas).
You are awesome.. I had an existing WIN 2016 Server VM and installed the DHCP service. Created the IP helper on my ICX6610 and I am now able to get Sonos working on the switch. You just made my weekend. Thank you so much for all your knowledge and willingness to help everyone. Kudos to you.
For anyone trying to figure out why the web interface won't load anything but the port map, be sure to take a look at the getting started guide! I lost quite a bit of time because I didn't flash the newest firmware from the get-go.
I have multiple VLANs in my home network LAN(Wired)/Wifi/IOT/DMZ etc. and primary reason for separating hosts into separate VLANs is to isolate them from each other for security reasons. I use firewall in pfSense to allow/restrict traffic between VLANs as well as to/from outside.
When you guys use L3 switch to do inter-VLAN routing do you utilize ACLs to filter traffic flow or do you allow everything. I am trying to better understand when it is beneficial to switch to using L3 switch for that purpose instead of a firewall. Thanks!
When you guys use L3 switch to do inter-VLAN routing do you utilize ACLs to filter traffic flow or do you allow everything. I am trying to better understand when it is beneficial to switch to using L3 switch for that purpose instead of a firewall. Thanks!
If I was at work, or at a client site, ACLs, lots of ACLs. The default policy between networks should be default deny permit by exception...
That said, at home. I am lazy, I permit all traffic between networks.. I will be adding a security camera vlan (rash of car break ins and people stealing off peoples docks on the water side), which I will isolate. I might do a more appropriate secure configuration as I renovate my house and put drops into each room and redo everything.
@kapone @fohdeesha
Does LLDP/CDP not work on the 40GB to 10GB with breakout cables? I have no issues with it on the gig ports, but after I put in the dualport 10G NC523 into my VM server replacing the quadport gig intel car, I am no longer getting the data from the switch on the distributed vmware switch.
As far as I can tell, I have it enabled (I did issue lldp enable ports all).
Does LLDP/CDP not work on the 40GB to 10GB with breakout cables? I have no issues with it on the gig ports, but after I put in the dualport 10G NC523 into my VM server replacing the quadport gig intel car, I am no longer getting the data from the switch on the distributed vmware switch.
As far as I can tell, I have it enabled (I did issue lldp enable ports all).
On a complete side-note, anyone know whether it's possible to configure a ConnectX-2 VPI card to connect to the Brocade via the 40Gbps connection and be configured in >10Gbps mode?
<EDIT>
I just figured it out. The ConnectX-2 VPI card only supports 10Gb over Ethernet and 40Gb over IB. The ConnectX-3 VPI card supports 40Gb over Ethernet. Well, ebay, here I come!
<EDIT>
I just figured it out. The ConnectX-2 VPI card only supports 10Gb over Ethernet and 40Gb over IB. The ConnectX-3 VPI card supports 40Gb over Ethernet. Well, ebay, here I come!
Last edited:
search ebay for 649281-B21 - they go for around $35, then you can flash the 40gbE firmware using this post: https://forums.servethehome.com/ind...net-dual-port-qsfp-adapter.20525/#post-198015
I can confirm that after a firmware flash, the HP cards work GREAT. And they are cheap as...
I bought three recently for $20/ea.Odd question: re Brocade 6450-24p. I am just now filling my first-ever Server Rack Cabinet (12U, xRackPro2) which has front and rear doors (and side doors). The Brocade 6450 is top of rack, Rear.
Cable management question: is it 'safe' to feed Cat6 cables on top of the Brocade? There is about a 1-2" clearance above the top-of-rack Brocade; and about 5-6 Cat6 cables would be resting on top the Brocade.
I don't think heat from the brocade would be a problem; but I like to ask before doing stuff. thanks for helping a server-rack noob out
Cable management question: is it 'safe' to feed Cat6 cables on top of the Brocade? There is about a 1-2" clearance above the top-of-rack Brocade; and about 5-6 Cat6 cables would be resting on top the Brocade.
I don't think heat from the brocade would be a problem; but I like to ask before doing stuff. thanks for helping a server-rack noob out
yeah there's nothing wrong with that
also thanks to @seatrope I realized the ICX6610 guide has been totally missing the "factory set-default" command, so anyone who had a used switch with a previous password was left to google when the guide told them to "enable" and it asked them for a PW they didn't have. The ICX6450 guide has had this instruction, not sure how I missed it on the 6610
added it - add missing password removal command · Fohdeesha/lab-docu@bc8d651
loading up for the LAN party event this weekend: https://i.imgur.com/F4z8ecE.jpg
will try to post setup/event pics during this weekend. I think we may be the first large scale LAN party to offer 10gbE ports for attendees at every table, even 40gbE depending on which table you sit at
mostly ICX6610s, some ICX7250's, ICX6450s, and a couple FCX's for WAN agg to fiber in the telecom closet
will try to post setup/event pics during this weekend. I think we may be the first large scale LAN party to offer 10gbE ports for attendees at every table, even 40gbE depending on which table you sit at
mostly ICX6610s, some ICX7250's, ICX6450s, and a couple FCX's for WAN agg to fiber in the telecom closet
Have a great LAN party, Guiness reports largest ever was an insane 22,180 people in Sweden, 2013loading up for the LAN party event this weekend: https://i.imgur.com/F4z8ecE.jpg
will try to post setup/event pics during this weekend. I think we may be the first large scale LAN party to offer 10gbE ports for attendees at every table, even 40gbE depending on which table you sit at
mostly ICX6610s, some ICX7250's, ICX6450s, and a couple FCX's for WAN agg to fiber in the telecom closet
Largest LAN party
I see Guiness records for largest amount of people, highest elevation, longest in time; but nothing about "First Ever LAN Party with 10gbE and 40gbE"
Thanks so much @fohdeesha for all your help. Greatly indebted. Icx 6610 humming along happily!
One dumb thing I did which would be obvious to most, but had me perplexed for a while. When I moved everything over from my old switch I ended up plugging in most things for a quick test, including several 10G DACs.
Soon after, everything on network started going crazy, unresponsive. PCs attached to 10G links started intermittently hanging. I realized that happened after I inserted the 10G DACs. On further investigation, of course I had looped a 10G cable and probably caused a broadcast storm...Duh.
One dumb thing I did which would be obvious to most, but had me perplexed for a while. When I moved everything over from my old switch I ended up plugging in most things for a quick test, including several 10G DACs.
Soon after, everything on network started going crazy, unresponsive. PCs attached to 10G links started intermittently hanging. I realized that happened after I inserted the 10G DACs. On further investigation, of course I had looped a 10G cable and probably caused a broadcast storm...Duh.