Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[fohdeesha]

Well that puts a damper on my desire to hunt down a 7750, assuming I still have my 7150s and 7450 when v9 becomes recommended GA. I was thinking of a 7750-26Q (or a -48F since is has 6x QSFP, but the additional 6x QSFP module tends to be like $500), so would a Arista DCS-7050QX-32S be a better choice? I can certainly find them for a lower price than the handful of 7750-26Q's I see on eBay. I'm going to need at least 6, maybe 8x 40G and 40G to 4x10G breakout support since I'll have more 10G devices than my 7450 will. The 7650 and 7850, when they've been available, have been waaaay out of my price range.

7050QX-32S are great switches - full x86_64 management plane, the 32S variant (versus the plain 32) can run the absolute latest EOS, and EOS is great to work with. prefer it to fastiron. two or three times as much RAM, can stick SSDs in them, can run container and KVM vm's on them, etc

 

[i386]

(Eos has commands to run the fans at 30%, that alone makes it awesome for a homelab )

 

[Caennanu]

so . . . i have a 6450-48P.
And, i have just received my 10G RJ-45 SFP module (ubiquiti UF-RJ45-10G).
But it seems my cable isn't stable enough to support it, as the link goes up and down. (don't have the ability right now to replace cable due to having to open up the driveway)
At 1Gb it is stable. So wanted to try at 2.5 and 5Gb, but i do not seem to have that option.
Is that option even available?

 

[tjk]

7050QX-32S are great switches - full x86_64 management plane, the 32S variant (versus the plain 32) can run the absolute latest EOS, and EOS is great to work with. prefer it to fastiron. two or three times as much RAM, can stick SSDs in them, can run container and KVM vm's on them, etc

What variant of Arista would you recommend over the 7750-48F, that is current and can run latest EOS?

 

[ZFSZealot]

so . . . i have a 6450-48P.
And, i have just received my 10G RJ-45 SFP module (ubiquiti UF-RJ45-10G).
But it seems my cable isn't stable enough to support it, as the link goes up and down. (don't have the ability right now to replace cable due to having to open up the driveway)
At 1Gb it is stable. So wanted to try at 2.5 and 5Gb, but i do not seem to have that option.
Is that option even available?

If you look back a few pages in this thread someone was talking about successfully using a 2.5G SFP+ transceiver in what I think was a 6610. It's about the transceiver.

 

[Caennanu]

If you look back a few pages in this thread someone was talking about successfully using a 2.5G SFP+ transceiver in what I think was a 6610. It's about the transceiver.

Right, found it and gotcha.
No support for autosensing, gotta 'hardwire' lower speeds.
oh well

 

[LodeRunner]

What variant of Arista would you recommend over the 7750-48F, that is current and can run latest EOS?

Maybe the 7050SX? The 7050QX-32S supports breakout; based on prices on eBay it would be cheaper, even accounting for needing passive optical breakouts and a patch panel.

Here's an EoS/EoL sheet which lists replacement models: End-of-Sale-Quick-Reference-Guide.pdf (arista.com)
That appears somewhat out of date, so here's Arista's EoS notices: Advisories & Notices - Arista

 

[Kevlar]

Official noob man here. I am having trouble with the transfer timing out when I attempt to install the new bootloader on my 6610 from the tftp server. I went through the whole thread and found one other dude who had the same issue but it was because his firewall was enabled. I already tried doing it with my firewall disabled (as that fixed it for him) and, son of a biscuit, it still didn't work. I have tried multiple ethernet cables for the connection from the management port to the network and from my laptop to the network but to no avail. I'll try another cable each tomorrow and see but I'm doubting that is the fix at this point.

I saw someone else who had a kind of similar issue and it was because they had some sort of janky digit-ery where their mac address should have been, so I checked mine and it seemed (to my untrained eye) to be formatted correctly for a mac address. I'm really hoping this is just some stupid thing I've overlooked but I just can't figure out what that might be at this point.

 

[LodeRunner]

Official noob man here. I am having trouble with the transfer timing out when I attempt to install the new bootloader on my 6610 from the tftp server. I went through the whole thread and found one other dude who had the same issue but it was because his firewall was enabled. I already tried doing it with my firewall disabled (as that fixed it for him) and, son of a biscuit, it still didn't work. I have tried multiple ethernet cables for the connection from the management port to the network and from my laptop to the network but to no avail. I'll try another cable each tomorrow and see but I'm doubting that is the fix at this point.

I saw someone else who had a kind of similar issue and it was because they had some sort of janky digit-ery where their mac address should have been, so I checked mine and it seemed (to my untrained eye) to be formatted correctly for a mac address. I'm really hoping this is just some stupid thing I've overlooked but I just can't figure out what that might be at this point.

What TFTP server are you using? Do you have a second computer where you can setup a TFTP client and test that server? If you do a netstat to show UDP ports, is port 69 bound to a physical interface or loopback? On Windows this would be "netstat -an -p udp | findstr 69" for most Linux "netstat -aun | grep 69" should work.

 

[ossicones]

I have four 6610 switches - I am installing two in one rack and two in another rack that is roughly 8-10m away. The question is, should I stack them, or trunk them together?
In the stacking guide, for the 6610 specifically, it states the max length DAC cable is 5m, with no specification for an optical option. I'm guessing this means I cannot stack them using fiber, meaning I would have to trunk them together. Can anyone speak to stacking over distances?

If I trunk, can I use fiber out the QSFP+ ports to get two 40GB links between switches?

 

[berezovskyi]

Hello folks. I have ICX6450-48P (firmware ICX64R08030u) and it seems that periodically crashes. I can't find any pattern for this. Previously it crashed sometimes and I added a big noctua fan to cool it down (`show chassis` showed pretty low temp) and crashes went away for more than a month.
Yesterday it came back. Today I had a chance to get the console cable plugged in and I saw a message that the crash was triggered by the hardware watchdog.
The device configuration wasn't changed for a long time ago. No new devices were connected to the network.

P.S. I found a command `#dd watchdog disable`. Will try it while the console cable is still connected.

 

[LodeRunner]

I have four 6610 switches - I am installing two in one rack and two in another rack that is roughly 8-10m away. The question is, should I stack them, or trunk them together?
In the stacking guide, for the 6610 specifically, it states the max length DAC cable is 5m, with no specification for an optical option. I'm guessing this means I cannot stack them using fiber, meaning I would have to trunk them together. Can anyone speak to stacking over distances?

If I trunk, can I use fiber out the QSFP+ ports to get two 40GB links between switches?

From the ICX6610 manual

ICX 6610 devices have two 40-Gbps QSFP stacking ports and two 4 x 10-Gbps QSFP stacking ports
on the rear panel. These ports can perform data transmission directly through copper links of up to 5
meters. ICX 6610 devices also support 40G-QSFP-SR4 at distances up to 100 meters

Also from the manual

Use 1 meter passive copper QSFP stacking cables or SFP+ fiber-optic cables to connect ICX 6610
devices in a traditional stack. 40G-QSFP-SR4 is also supported.

I'm thinking that when the 6610 came out, DACs might not have exceeded 5 meters. I don't know what spec covers DACs or if it initially had a limit of 5m that was then extended to 10m. So the -SR4 optic is called out, but with no length limit. If you have DACs or optics and 10m cabling, it's worth testing.

 

[lilkeeney]

I am trying to use my 6450-48P as a POE injector. I have a cellular modem in the attic that is POE powered plugged into one port of my switch and then I have another port plugged into the WAN2 port of my router. These two ports are on a seperate VLAN that nothing else is on. This works fine most of the time. However, if one gets disconnected it doens't always reconnected without a reboot. I have a feeling that some settings may be interfering with this. Could Flow Control or STP be an issue here? Any thoughts on how to dumb down the switch and act just like a POE injector? And yes, I could just use a POE injector, but this helps with the wiring and allows me to reboot the cellular modem if needed remotely.

Here are the settings and diagnostics for the cellular modem.




And here are the configurations and stats for the WAN2 port on my router.

 

[Kevlar]

What TFTP server are you using? Do you have a second computer where you can setup a TFTP client and test that server? If you do a netstat to show UDP ports, is port 69 bound to a physical interface or loopback? On Windows this would be "netstat -an -p udp | findstr 69" for most Linux "netstat -aun | grep 69" should work.

I'm just using the one straight from the zip that Fohdeesha made and pointing it to the correct folder replacing the ip address in the instructions with my tftp server ip address. Tested it on a second windows pc and had the same issue where it opens up the file transfer window 4 times and then times out on each of them after not transferring anything.

When I enter that netstat command I get:
UDP 0.0.0.0:69 *:*

....never mind, I was in the middle of getting ready to send this reply and I found that I either needed to select something other than the local (127) server interface in the tftpd64 app or I needed to use a different ip address than I was previously using for the switch because it just worked. Thanks for your quick response!

 

[Cncjerry]

hello all, my 6610 arrived today and thanks to the great write-up it is up, licensed and operational.

Thanks again for the great work.

Jerry

 

[S!rius]

Hello together

Been following this thread for a while and now got myself a 7250-48P. Now before I do my network design I have a trivial question: Can I use the 8-uplink ports as "normal ports" to connect end devices. Must confess that I have not read all 315 pages. Maybe someone of you has asked this or a similar question. The search did not bring any results.

 

[Freebsd1976]

Hello together

Been following this thread for a while and now got myself a 7250-48P. Now before I do my network design I have a trivial question: Can I use the 8-uplink ports as "normal ports" to connect end devices. Must confess that I have not read all 315 pages. Maybe someone of you has asked this or a similar question. The search did not bring any results.

yes

 

[klui]

Uplink is the same as a "normal" port.

 

[Jason Antes]

I'm just using the one straight from the zip that Fohdeesha made and pointing it to the correct folder replacing the ip address in the instructions with my tftp server ip address. Tested it on a second windows pc and had the same issue where it opens up the file transfer window 4 times and then times out on each of them after not transferring anything.

When I enter that netstat command I get:
UDP 0.0.0.0:69 *:*

....never mind, I was in the middle of getting ready to send this reply and I found that I either needed to select something other than the local (127) server interface in the tftpd64 app or I needed to use a different ip address than I was previously using for the switch because it just worked. Thanks for your quick response!

I had it do something like that to me too, pretty sure it was because I had the TFTP set to the wrong folder as a source. It was either down 1 level too far or up 1 level too far.

 

[mbze430]

I can license them, just be aware the copper version (the vdx6740t) draws significant power, it's first gen 10gbase-T PHY, so like a few watts per port. I think they idle around 180w but I don't have any numbers written down from when I had one. also not quiet

Do you have another recommendation, but cheap? I need 10GBase-T ports with 40Gbe or higher to uplink to a DX010 I was looking at a VDX6740T but that power draw is a bit insane, I looked it up in the data sheet from Extreme Network