Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Well, I'm glad I bought it. The $75 switch not only looks brand new, it actually has two ICX7400-4X10GF 4-port 10-Gigabit SFP+ modules in it! There is one in the back replacing a 40GbE output. I've had good luck on eBay lately I guess!
I don't need all the switches I have:
ICX6450-24P
ICX7450-32ZP
ICX7450-24P
ICX7150-C12
I'm definitely keeping the ICX7450-32ZP that's the one I have been lusting for a long time. I definitely selling the ICX7150-C12 as they are going for huge money on eBay now and I don't need it in the new house.
The mental debate is between keeping the 6450 or 7450 as a spare or for future expansion. I'm inclined to think the 7450-24P is better to keep as a spare especially since it's the same switch I am using as my primary, minus the eight 2.5GbE ports. But I LOVE the low power and reliability of the 6450. I've had the 6450 running for +6 years without any issues ever. It's a rock, sips power, and if I ever need to can run it over PoE like fohdeesha points out in the first two post to this thread.
Decisions, decisions. What a first world problem.
Last edited:
Ok, tried to update my 7250-48p today from 9010jcd1 to 9010jcd2 and it didn't work right, it booted to the OS prompt via either that image or the secondary jcd1 image. I ended up having to go back to 8095m and then update that again to 9010jcd2 to restore operation. Any ideas? the initial update seemed to go right until it didn't reboot.
I finally made the call last weekend to e-waste a bunch of old server "stuff" that was cluttering up my SMALL rack and hobby room. Three old Dell servers (R610, R720 and R630) along with my old Aruba switches (two 48 port S2500, a 24 port S2500 and a 48 port S3500) along with some other misc. "stuff". All of it had hit that horrid place of being worth less than it would cost to ship.
My two ICX 6610-48s have been soldering on for the last few years doing a great job.
Yeah. Out of space to store it and I wouldn't be using it again. After checking the actual sell prices on eBay plus cost of packing materials plus cost of shipping, it just wasn't worth keeping.
I thought of posting a "come and get it!" note on here, but unfortunately, I hit the "this crap has GOT TO GO! And NOW!" mode and loaded it up!
I still have my R740 and C6400 with 4 XC6420 nodes in the lab along with the "backup" R730.
And I probably had a harder time getting rid of that R610. That was my first REAL server and my brother got that for me as, well, e-waste, from his hospital's data center.
Seriously, I'm pouring one out over here for that 630.
Sorry for replying to such an old post. I currently have two ICX 6610s setup following your post, so 1/2/1 and 1/2/6 (garage) stacks with 2/2/1 and 2/2/6 (server rack) and I have been using the remaining ports broken out into eight 10Gbe ports for my servers in the rack. And it has worked perfectly for the last 2 or 3 years or so.
Now, I would like to add a SX6036 to my network mix. Originally, I was going to leave it completely isolated from the ICX networking (Proxmox Ceph networking only). However, I have decided the system would be more flexible for future upgrades if I can connect the ICX to the SX with a pair of 40Gbe connections and then use the 4x10Gbe splits from the SX to go to my servers.
The problem is I don't ever remember seeing a sway to set the other two non stacking ports on the ICX 6610 to 40Gbe mode. Am I hosed, or just missing a bit of information (that has probably been covered in hte 490+ page ICX master thread!)?
Thanks!
Clint
PS: It isn't as simple as removing the MPO->4xLC cable and install an MPO-MPO cable? I would think there has to be some command to "collapse" ports 1/2/2-1/2/5 into a single 40Gbe port...
Last edited:
Why don't you modify @fohdeesha's recipe and use the broken out ports as stacking ports instead? Per post #4 of this thread you can't "no breakout" the 4x10G ports in slot 2--no such command exists on the 6610.
491 pages. And the answer is on page 1! Thanks. I'm fairly sure there is no way to "stack" the ICX 6610 with the SX6036 (that would be REALLY cool by the way!) so I was trying to find a way to use the other two QSFP ports to link the two switches together at 40Gbe.
I might need to revert back to plan A.
Stacking ICXes with MSXes would only work in fantasyland. Stacking on the ICXes and other vendors' switches are limited to a few models or in some cases only 1 model.
If you can stack your 6610s using the 4x10G ports you only need to connect one of its 40G ports to the 6036 in order for them to talk to each other, unless you want to use LACP.
If you can stack your 6610s using the 4x10G ports you only need to connect one of its 40G ports to the 6036 in order for them to talk to each other, unless you want to use LACP.
Looking into the media types I can see that both my 10G optics are being detected as only 1G optics.
Code:
Port 1/2/2: Type : 1GE M-SX(SFP)
Port 1/2/3: Type : 1GE M-SX(SFP)If it helps this is the EEPROM dump provided by running #dm optic 1/2/3 eeprom
Code:
beefchunk#dm optic 1/2/3 eeprom
0000 03 04 22 00 00 00 00 00 08 00 00 06 67 00 00 00 ..".........g...
0010 00 00 01 00 4d 69 6b 72 6f 54 69 6b 20 20 20 20 ....MikroTik
0020 20 20 20 20 00 00 40 20 53 2b 52 4a 31 30 20 20 ..@ S+RJ10
0030 20 20 20 20 20 20 20 20 32 2e 31 36 05 00 00 b0 2.16....
0040 02 1a 00 00 48 44 31 30 31 4e 31 36 4a 48 44 20 ....HD101N16JHD
0050 20 20 20 20 32 32 30 39 32 39 20 20 68 00 06 4b 220929 h..K
0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
Last edited:
@lawrence1986 I saw earlier in the thread you were having a similar issue with these Miktotik S+RJ10 optics.
I tried changing the flow-control and setting it manually to 10G-full but that doesn't work as its only being detected as an SFP.
Did you do anything else to get them to work?
I tried changing the flow-control and setting it manually to 10G-full but that doesn't work as its only being detected as an SFP.
Did you do anything else to get them to work?
So long as I can use the existing MPO cables to stack the two ICX 6610 switches, I don't care which of the ports are used for stacking. My understanding on the 6610 is that ports 1/2/1 and 1/2/6 are dedicated stacking ports, so even if I did stack with 4x10G ports, I still couldn't use the two (or even one) of the 40G ports to link the SX6036 to the ICX 6610s.
If that is a mistaken assumption, I would love to hear how to set it up.
At this point, I am leaning towards leaving the ICX 6610s stacked the way they are, use LACP and link a 4x10Gb break out cable from the SX6036 to four of the 1/3/x ports on the 6610. At that point, all of the high speed traffic between nodes will be going through 6036 (as well as all the VM traffic). The ICX switches will be handling all the 1Gbe traffic and internet routing. (which is only 800MB down).
This would leave all the high speed traffic (cluster management, Ceph, backups, etc.) on the 40Gbe network and the slower stuff passing between the switches on a 4x10Gbe LACP which is MORE than enough capacity for a home lab.
I think the answer is 'No' but maybe I'm missing something.
Is there a way to see the optical information on the 40g stacking ports on an ICX6610?
I've got a kaiam xqx2502 40g-lr4 module that works happily to move traffic but I can't get any status from it. Surprised that "stacking across racks via fiber" isn't a supported configuration for getting media info, would seem like a reasonable use for it. show media calls it '40GBASE-Passive Copper'. In this case it's not being used to stack, but as a 40g uplink and I have 'stack disable' in the config. Testing it in 1/2/2 to 1/2/5 breakout port gives no error but has no information for show optic either.
running 08.0.30uT7f3. I think it has the full unlocks from fohdeesha: ICX6610-10G-LIC-POD, ICX6610-ADV-LIC-SW and ICX-MACSEC-LIC
E: I think I can answer my own question, from fohdeesha's documentation on the 6610:
Is there a way to see the optical information on the 40g stacking ports on an ICX6610?
Code:
SSH@coreswitch#show optic 1/2/1
port 1/2/1 is a stacking port and not capable of digital optical monitoring.running 08.0.30uT7f3. I think it has the full unlocks from fohdeesha: ICX6610-10G-LIC-POD, ICX6610-ADV-LIC-SW and ICX-MACSEC-LIC
E: I think I can answer my own question, from fohdeesha's documentation on the 6610:
I think that means you have to use brocade/foundry optics. I misunderstood the 'by default' and thought you had to enable it for other brands. It's a bit confusingly worded.
Last edited:
So question here, newb probably, i have an ICX6450 on the way to add to my system. It's for an outbuilding, will be on OM3 fiber, connected to my 6610 in the basement. The outbuilding will have just a few devices on it, cameras, wifi ap, and a backup server. Should i be trying to trunk the 6450 to the 6610, or is it unnecessary complication for this application? There are a few vlans on my system, trying to keep camera and video traffic separated from more basic stuff. I've never done much more than dumb switches before now, and had no issues with switch to switch. Loving the 6610, assuming the 6450 will be more of the same.
If I have a stack of four or more 6450-48p in a single ring topology (two sfp ports per switch) , if one or more switches in the ring dies will the connection still pass through the dead switch, or does it become a daisy chain, or could any isolation issues arise? I couldn’t find information on this in the manuals. Thanks
Data will not pass through an inoperable switch. If one switch is inoperable, the remaining switches will be able to communicate with each other; if two switches fail and there is a switch between them, that switch will be isolated.
I am running a quorum of opnsense firewall minitowers equipped with silicom pass through Ethernet cards in front of my 4 x 6450 - 48p stack, passing authorized traffic to a designated VLAN on the 6450 stack. Is it possible to set the 6450 stack to only accept traffic authorized by opnsense?
The idea is if any of my opnsense servers power down due to failure or for maintenance, pass through ethernet will still continue to forward existing traffic to the 6450 stack. I want the stack to only allow passthrough traffic that had previously been authorized by my opnsense servers prior to shutdown and discard unauthorized traffic. If this is possible what will I need to do/what terms do I need to look up?
The idea is if any of my opnsense servers power down due to failure or for maintenance, pass through ethernet will still continue to forward existing traffic to the 6450 stack. I want the stack to only allow passthrough traffic that had previously been authorized by my opnsense servers prior to shutdown and discard unauthorized traffic. If this is possible what will I need to do/what terms do I need to look up?
Hopefully this thread can help me out. I currently have a VDX6740T and I'm thinking about getting a ICX6610-24P to replace it. However, I have a few questions.
- Which is louder, VDX or ICX?
- Which would have the lower power draw using 14 ethernet ports and 2 40Gb ports? On the ICX, 3 of the ports would be outputting PoE.
- Which runs hotter using the same ports as in question 2?
- Which one would be "better" overall? I'm not worried about learning a new OS so that isn't an issue.