Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
did you not just follow the bolded update/config guide in my OP? it gets you up to date, the switch configured in a "dumb" l2 only switching mode with an IP to manage it,, but running the router firmware so if you want to play with routing features at a later point you can. It's kind of silly to run the switch only image code, and in fact they're being killed off in the 7 series soon as the router image provides all the same features and more
@fohdeesha - Thank you so much! Finally got my ICX-6610-24PE up and running, noise level thankfully isn't a problem (I have a 18U network cabinet with doors)
Amazing that I have my ESXi server connected to the switch with a $9 NetApp QSFP+ cable @ 40 Gbit !! woohoo !!!
I also used an Ipolex 10GBase-T SFP module and I'm doing 10 GbE over Cat5E
Does the switch support 24V Passive POE? Some of my devices are 24V passive, looks like I will be buying some 802.3af/at compatible cameras
Amazing that I have my ESXi server connected to the switch with a $9 NetApp QSFP+ cable @ 40 Gbit !! woohoo !!!
I also used an Ipolex 10GBase-T SFP module and I'm doing 10 GbE over Cat5E
Does the switch support 24V Passive POE? Some of my devices are 24V passive, looks like I will be buying some 802.3af/at compatible cameras
No, it doesn't. "Passive" PoE is not a standardized one, and each vendor has its quirks added to it. The only interoperable between vendors and equipments for sure is the 802.3af/at.
Question for the folks here who know more than I do about breakout cables -- and that would be all of you, because I know very little.
I'm shopping for a pair of 40Gb-to-4x10Gb breakout cables to use with my brand, spankin' new (to me) ICX 6610.
With respect to passive DAC versus active optical cables: what, if any, are the advantages/disadvantages of one over the other?
I have my eye on these Finisair optical cables, 'new' at $30 apiece:
And also these used Arista DAC cables at $29 each:
Later: Well, after a little light googling I've answered my own question: the active optical cable will use more power and work at a greater distance, while the passive copper uses less power and is only good for short runs. So, DAC it is!
I'm shopping for a pair of 40Gb-to-4x10Gb breakout cables to use with my brand, spankin' new (to me) ICX 6610.
With respect to passive DAC versus active optical cables: what, if any, are the advantages/disadvantages of one over the other?
I have my eye on these Finisair optical cables, 'new' at $30 apiece:
Finisar Quadwire 40G QSFP 4x10G SFP Fan-Out Cable FCBN 510QE2C02 2M NEW @3 | eBay
Find many great new & used options and get the best deals for Finisar Quadwire 40G QSFP 4x10G SFP Fan-Out Cable FCBN 510QE2C02 2M NEW @3 at the best online prices at eBay! Free shipping for many products!
www.ebay.com
And also these used Arista DAC cables at $29 each:
Arista+Original+2m+DAC+QSFP+40gb+to+4+SFP+10gb+Breakout+Cable+Cab-q-s-2m for sale online | eBay
Find many great new & used options and get the best deals for Arista+Original+2m+DAC+QSFP+40gb+to+4+SFP+10gb+Breakout+Cable+Cab-q-s-2m at the best online prices at eBay! Free shipping for many products!
www.ebay.com
Later: Well, after a little light googling I've answered my own question: the active optical cable will use more power and work at a greater distance, while the passive copper uses less power and is only good for short runs. So, DAC it is!
Last edited:
A benefit of an AOC is they are much slimmer and allows easier cable management.
Yes, they are essentially a fiber optic cable with the optics "glued together" in a single piece.
Although the cable is slimmer, you have to fiddle around with the entire SFP connector when moving those things around, partially negating the advantages of a slim cable.
Our racks in our lab at work have a multitude of mini-SAS HD and thin and thick passive copper DACs which make maintenance a bear. Rerouting an AOC's transceiver is welcomed compared to getting at a component blocked by those aforementioned monstrosities.
I got a 6450-24p two months ago (thanks @fohdeesha for the licenses and for recommending and "supporting" the ICX switches ).
It's a far better solution than that unmanaged 8port switch that I used for the ipmi/managemange ports in my serverrack.
It's stable, there are free firmware/software updates and I'm happy with it
).It's a far better solution than that unmanaged 8port switch that I used for the ipmi/managemange ports in my serverrack.
It's stable, there are free firmware/software updates and I'm happy with it
IMHO, if you have those handy then they cause no harm being put in; if you don't have those, unless your equipment lives in a very extreme environment, they don't make a big difference in dust collection inside the switch. The same damages (or worse) will occour also on the metal part of the RJ45 ports.
I bought an ICX6450 recently and I want to start making use of the SFP+ ports for my server.
Here's the question, can I use a QSFP+ to 4xSFP+ breakout DAC on my server end and connect up only some of the SFP+ connectors to the switch and get them to work at something like 20/30Gbit speeds? It's either this or I'll need to get a QSFP+ to SFP+ adapter and use just an SFP+ DAC.
I need at least 1 port for a 10Gbase-T transceiver for my computer in another room (I have the house wired with Cat6a).
EDIT: After looking a bit more, it seems that NICs don't typically support breakout cables; the one I'm specifically looking at getting is a ConnectX-3 rebrand (the HP 649281-B21) and Mellanox says none of their NICs support it with specific mention of the ConnectX-3 as an example. So it looks like an adapter and SFP+ DAC are the only options (until the switch bug bites again and I upgrade to a model with a QSFP port).
Here's the question, can I use a QSFP+ to 4xSFP+ breakout DAC on my server end and connect up only some of the SFP+ connectors to the switch and get them to work at something like 20/30Gbit speeds? It's either this or I'll need to get a QSFP+ to SFP+ adapter and use just an SFP+ DAC.
I need at least 1 port for a 10Gbase-T transceiver for my computer in another room (I have the house wired with Cat6a).
EDIT: After looking a bit more, it seems that NICs don't typically support breakout cables; the one I'm specifically looking at getting is a ConnectX-3 rebrand (the HP 649281-B21) and Mellanox says none of their NICs support it with specific mention of the ConnectX-3 as an example. So it looks like an adapter and SFP+ DAC are the only options (until the switch bug bites again and I upgrade to a model with a QSFP port).
Last edited:
I've got an ICX-6610-48P on the way along with some mikrotik S+RJ10 SFPs that I'm planning on using to create a 10G link over CAT6 to a Mikrotik CRS305-1G-4S+IN that will sit on my desk. The final piece of my puzzle is the link from the switch to the desktop. From this thread it seems like the best bet is a ConnectX-3 (is that still the case?). I'm just not sure what I need to buy for the QSFP (??) to Mikrotik. I'd prefer to go fiber. If anyone could point me to the correct parts and cable I need to do 10G from the Mikrotik to the Desktop I'd appreciate it.
QSFP+ and SFP+ are different standards.
SFP/SFP+ is smaller and does ~ 10GbE.
QSFP+ is bigger than SFP/SFP+ and can do 40 ~ 56GbE
Both can support direct attached cables or transceivers. And transceivers can be optical or copper. But your Mikrotik doesnt support QSFP, only SFP
If you wish to buy a ConnectX-3 card (that has two QSFP slots) you would need a QSFP to SFP adapter: Mellanox MAM1Q00A-QSA-SP (Single Pack MAM1Q00A-QSA) Ethernet Cable Adapter 40Gb/s to 10Gb/s QSFP to SFP+ and you can then use a copper transceiver on both ends: https://www.fs.com/products/87588.html or Cisco SFP-10G-T-S Compatible 10GBASE-T SFP+ Transceiver Module - OR you can use an optical transceiver: https://www.fs.com/products/65334.html and cable: https://www.fs.com/products/41730.html (available in different lengths)
Just make sure the optical transceivers you select are compatible with the Mikrotik switch and with the Mellanox card
I bought two Connect X3's (HP) and flashed them. Have QSFP+ -> SFP+ tranceivers from Ebay connected to my switch via DAC, and a QSFP+ Amazon cable as a peer-to-peer DAC connection between the two computers.
The Connect X3's are the way to go, imo. Or something newer that is SFP+ only since 40gbe seems to be a dead tech.
The Connect X3's are the way to go, imo. Or something newer that is SFP+ only since 40gbe seems to be a dead tech.
It's just a lot of acronyms that are very confusing. Kinda like 802.11g, a, n, ac, ax. They should really rename them.
Because the SFP and QSFP form factors are common place in data centers newer data rates have piggybacked on these two for newer standards when possible.
SFP (1G); SFP+ (10G); SFP28 (25G, 'cuz it maxes out at 28Gbps)
QSFP (4 x 1G); QSFP+ (4 x 10G; 40G [+56G for Mellanox]); QSFP28 (100G; 4 x 28G)
OSFP (400G)
QSFP-DD (400G)
They're backward compatible. For example, QSFP28 to SFP28 breakouts should (ours at work does) support 40G to 4 x 10G. But QSFP+/SFP+ most likely will not support 100G/25G/4 X 25G.
Just to be completely clear, the Mikrotik CRS305-1G-4S+IN you say you want to connect to your desktop only has SFP+ ports on it. It does not have QSFP or QSFP+ ports on it, so those aren't relevant.
You have these choices as I see it:
1a) Another copper run to a copper card:
* A copper 10gbe card (I've been using Intel X540-T2s, and even used on EBay where I get them, they are pricey.)
* Another mikrotik S+RJ10 or similar for the SFP+ port on the CRS305-1G-4S+IN.
* A Cat6a copper cable
1b) Another copper run to an SFP+ card:
* A dual or single port SFP+ card. (see 2a below for details)
* A pair of mikrotik S+RJ10 or similar for the SFP+ port on the CRS305-1G-4S+IN and the SFP+ NIC card
* A Cat6a copper cable
2a) A fiber run:
* A dual or single port SFP+ card. I've been using Solarflare SFN7122Fs which go for $30-$40 on Ebay, but there's also the ConnectX-3 which I have no personal experience with, and there are surely others as well.
* Two 10GBASE-SR SFP+ fiber transceivers. I've been using the house branded ones from fs.com (Cisco SFP-10G-SR Compatible 10GBASE-SR SFP+ Transceiver Module), but I am certain you can save here if you shop around and know what you are doing. I'm just getting started on 10gb so I wanted something I knew would work. Probably $6-$20/transceiver, but the experts here should feel free to set me straight.
* An appropriate length of fiber cable. I've bought different lengths of these for short (<50 foot) fiber runs: OM4 50/125 LC-LC Multimode Fiber Optic Cable Duplex 5m (16ft) - Aqua. Again, you can probably save here, but I'm a novice so I went with fs.com to be sure I'd get something I was confident would work.
2b) A DAC run:
* A dual or single port SFP+ card. (same as 2a; see above.)
* A appropriate DAC (Direct Attached Copper) cable. For example, here's a 10ft cable from fs.com: Brocade 10G-SFPP-TWX-0301 10G SFP+ DAC Twinax Cable. This replaces both fiber transceivers and the fiber cable in 2a.
2a & 2b are the best choices in my book. If your switch is a good distance away in your room - say you have a large room - fiber may be the only practical option of the two - DAC cables are length limited. But if your CRS305-1G-4S+IN switch is very close to your desktop, the 2b (DAC) option is likely best since it will be cheaper and is very simple to understand and use. 1b may be a bit cheaper than 1a, but either will likely be more expensive than either 2a or 2b. I would do Fiber (2a) before I would do 1a unless I had to use copper for some reason, like for example, pre-existing Copper wiring.
In my server room I have been using a mix of all of these, and I'll admit to buying some stuff without cost being my overriding concern, just so I can see how it all works.
I find myself buying 3x the requisite number of required patch cables just so that I can make the rack look pretty
Or read the Great Deals section and blindly buy stuff off eBay This forum is VERY addictive for that