Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[rigel.local]

the 4 QSFP ports on the back are not locked. See post #4
they are however configured by default for stacking.
this post talks about a mix of stack and not stacking qsfp ports.
You can configure all 4 QSFP for no stacking and then use them to connect to hosts e.g. 2x 40Gbps and 8 x 10Gbps.

Thank you so much for explanation!!

On another note does anyone know what does "Premium software" vs "Base software" mean from "BROCADE ICX 6610 ORDERING INFORMATION" section in this document?

https://fohdeesha.com/data/other/brocade/ICX6610%20Datasheet.pdf

Practically speaking show we worry about loosing some features when buying ICX6610-24P-E vs ICX6610-24P-PE? "PE" models of 6610 are more expensive.

 

[DerTom]

This sort of thing is what VRFs were invented for. You create a VRF for each 'virtual routing table' you need, with the proper routes to external networks, and assign VLANs to the VRFs they need.

The issue here is not the existence of different gateways, it's the existence of different routing tables.

and @DerTom

Concur that VRF would solve this.

But, does the ICX6450 have VRF support?

AFAIK - I do not think VRF support is available on the ICX6450 (first post in the thread and here.)

Yes, the 6450 will not support VRF...

Is there another way to solve this?

If not, I will have to wait until an ICX7250-24p is within my budget...

 

[rigel.local]

Sorry for newbie spamming here. some crazy idea.

Has anyone thought about taking out all the internals from for example ICX 6610 and placing them in a generic 2U rack case with cutting out the front panel. And then using 3d printing print a custom panel with cutouts for this specific ICX 6610 ports arrangement. After that we can use bigger and less noisy fans that push more air.

Or included fans are custom and programmed in a way no other fans can be used with custom connections?

 

[Rttg]

Is there another way to solve this?

Do you need L3 routing on the switch itself? It looks like you could use pfsense to route based on L2 VLANs.

 

[MikeSpears]

I suspect I fixed this, but will know for sure later as my lab lives in a datacenter so gotta go back to test... I never disabled stacking on my switch.

Nope, didn't help.

 

[MikeSpears]

Nope, didn't help.

Can someone recommend a known working 40G to 4x 10G breakout cable? Ideally via Amazon so I can get it quickly..

 

[kapone]

Can someone recommend a known working 40G to 4x 10G breakout cable? Ideally via Amazon so I can get it quickly..

Confirmed working. (can't find em anywhere else except ebay)

Dell 40G QSFP+ to 4x10G SFP+ 3M DAC Splitter Cable DAC-QSFP-4SFP-10G-3M 27GG5 | eBay

Find many great new & used options and get the best deals for Dell 40G QSFP+ to 4x10G SFP+ 3M DAC Splitter Cable DAC-QSFP-4SFP-10G-3M 27GG5 at the best online prices at eBay! Free shipping for many products!

www.ebay.com

And wow, have they gone up in price! I bought 4 of these in 2018 for $50 shipped!

 

[MikeSpears]

Thanks. Might have to get one overnighted

 

[RandellH]

Hi All,

I have a ICX-7250-24 switch I followed the Fohdeesha Docs for setup and licensing. I ordered 3 Mikrotik S+RJ10 SFP modules for 10 G connectivity.When I plug the 10G transceivers into the switch they do not light up and when I type show media validation the Type is 1GE M-SX(SFP)
The sticker on the outside says MikroTik S+RJ10 is there anyway I can vailid the transceivers?

A little bit of a necro, but I had the same issue (I didn't read all the thread before I tried) with mine. I tried this module:
HiSense SFP-10G-SR from Amazon (Size: 10GBase-T: 30m)

Edit: What was received was an ASF-10G2-T.

It seems to work. I plugged my ATT gateway into the 5gb port (on the gateway) into this module and it lit up and I have internet. The switch shows a 10gb link and ATT gateway shows a 5gb link.

I ran speedtest and I was maxing the connection on both upload and download. I will probably try the 2gb service soon to see how that works.

SSH@switcheroo(config)#show media e 1/2/8
Port   1/2/8: Type  : 10GE SR 300m (SFP+)
             Vendor: OTHERS             Version: 02
             Part# : SFP-10G-SR         Serial#: XXXXXXXXXXXX

I noticed the part number above doesn't seem to be the one I ordered, so they are a bit careless with their programming that information.

SSH@switcheroo(config)#show interfaces brief e 1/2/8

Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/2/8      Up      Forward Full 10G   None  No  30   0   xxxx.xxxx.xxxx

SSH@switcheroo(config)#show media validation

Port       Supported Vendor               Type
----------------------------------------------------------------------
1/2/8      Yes       OTHERS                Type  : 10GE SR 300m (SFP+)

Edit: Looking at the Amazon page, it looks like it has a Marvel 88x3310, which might have some issues:
GBASE-T transceivers performance with NBASE-T - Marvell 88x3300 v/s Aquantia AQS-107 | ServeTheHome Forums

So, I might have some issues when I go to 2gb

 

[kapone]

The switch shows a 10gb link and ATT gateway shows a 5gb link

huh??

 

[RandellH]

huh??

This module, like the S+RJ10 basically turns the 10g port on the Brocade into a multigig port. Of course Brocade doesn't "see" that, it just sees that there is a 10gb SFP+ module. The ATT gateway has a port that supports 1gb, 2.5gb and 5gb. So, when I put that into this module, the module and ATT negotiate 5gb and Brocade is non the wiser.

 

[kapone]

This module, like the S+RJ10 basically turns the 10g port on the Brocade into a multigig port. Of course Brocade doesn't "see" that, it just sees that there is a 10gb SFP+ module. The ATT gateway has a port that supports 1gb, 2.5gb and 5gb. So, when I put that into this module, the module and ATT negotiate 5gb and Brocade is non the wiser.

I'm aware of modules that do that. But "HiSense SFP-10G-SR" doesn't seem to be something special. It's simply a transceiver?

 

[RandellH]

I'm aware of modules that do that. But "HiSense SFP-10G-SR" doesn't seem to be something special. It's simply a transceiver?

Perhaps I am just misinformed (good chance, since I'm new to this stuff). I was under the impression that you needed a special transceiver to get these switches to play nice with 2.5/5gb devices, otherwise it wouldn't work unless the both devices supported 10gb.

 

[kapone]

Perhaps I am just misinformed (good chance, since I'm new to this stuff). I was under the impression that you needed a special transceiver to get these switches to play nice with 2.5/5gb devices, otherwise it wouldn't work unless the both devices supported 10gb.

You're right.

But that's my point. If this is the one you bought, it shouldn't have worked. Maybe what you bought is actually a different model number?

 

[RandellH]

You're right.

But that's my point. If this is the one you bought, it shouldn't have worked. Maybe what you bought is actually a different model number?

Here is the one I ordered:
https://www.amazon.com/dp/B06XQBFHNL (Size: 10GBase-T: 30m)

And here is a picture (sort of matches the picture on the Amazon page, same part number but mine actually has the multigig text). Doesn't seem to match what the switch reports

From the switch console:
Type : 10GE SR 300m (SFP+)
Part# : SFP-10G-SR

This is a picture of what I plugged into that cage:


The point of my post was that both I and person I replied to got the Mikrotik S+rj10 wasn't even properly detected by the switch (no link light) and this one appears to work. (I admitted I didn't search enough and there was postings about that one not working)

I updated my original post, but it looks like according to Amazon page it has the Marvell 88x3310 which appears to have some issues according to another post here.

 

[kapone]

See? The model # is different!

 

[itronin]

Can someone recommend a known working 40G to 4x 10G breakout cable? Ideally via Amazon so I can get it quickly..

I have had 0 issues with the cisco AOC QSFP to 4xSFP+ breakout cables both 2m and 10m.

 

[RandellH]

See? The model # is different!

I might have ninja edited this after you initially read it:


While the switch "sees" SFP-10G-SR, it is ASF-10G2-T (according to the actual label on the product).

I should have posted that part number, or the link to the Amazon page. I was attempting to avoid posting an Amazon link. But now that I look at the Amazon page even closer, their picture shows ASF-10G-T. So all kinds of crazy.

So, I ordered their 10GBase-T: 30m, which has a picture of an ASF-10G-T but received a ASF-10G2-T and when installed the switch sees an SFP-10G-SR. That should clear it right up!

 

[MikeSpears]

Confirmed working. (can't find em anywhere else except ebay)

Dell 40G QSFP+ to 4x10G SFP+ 3M DAC Splitter Cable DAC-QSFP-4SFP-10G-3M 27GG5 | eBay

Find many great new & used options and get the best deals for Dell 40G QSFP+ to 4x10G SFP+ 3M DAC Splitter Cable DAC-QSFP-4SFP-10G-3M 27GG5 at the best online prices at eBay! Free shipping for many products!

www.ebay.com

And wow, have they gone up in price! I bought 4 of these in 2018 for $50 shipped!

Ordered, did overnight shipping so I should have it very quick.

 

[DerTom]

Do you need L3 routing on the switch itself? It looks like you could use pfsense to route based on L2 VLANs.

That is what I'm doing right now. I'm about to go to 10Gb and my pfSense-box does not have 10Gb nics.

By having to use another router/switch I'm also thinking about using an ICX6610-24p and 40Gb,There is just this little problem with a lot of watts being needed and a difficult task to get this switch being silent (fan-swap).