Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
657
239
43
Invest in some 3m gel filled quick splice connectors, they'll make splices like that really easy. Stripping wires that size is a pain, especially when they're stranded and not solid core.
 
  • Like
Reactions: Noppadet

Noppadet

New Member
Feb 1, 2020
8
1
3
US
For anyone who's trying to sort out how to connect to the mini USB port on the 7xxx series:

I wired up one of these a couple of days ago, it took me about 3 minutes. Pins 3 and 6 on a Cisco-compatible rollover cable are Rx and Tx, you just need to connect those to the white and green wires in a mini USB cable. Rx from the switch goes to Tx on the Cisco console cable and vice-versa.

View attachment 12955

#DongleLyfe

edit:

View attachment 12957 View attachment 12956

The pinout on your keystone jack might be different, just wire white to pin 6, green to pin 3 and black to pin 4 or pin 5.
Man I'm having seriously bad luck with a DIY jack. I tried this with a keystone and still nothing. I hope the serial port isn't fried. My serial cable is late because of the northeast weather and I'd really like to get this tested, but so far blank console (Xterm/Putty Windows, MiniCom/Putty Linux). Flow off, 9600 baud, 8 data bits, 1 stop bit.





With the keystone I plugged in a Cisco SERIAL auf RJ45 Cisco Systems Console AWM 20251 E129757-3 LL97744 | eBay stock cable (I reterminated the RJ45 connection), and went directly into a USB to RS232 connected to all different USB ports (2.0/3.0/3.1).

At least I'm learning something new.

Edit:
Yup. The actual 3M connectors are usually better quality but generics work just fine in a pinch.
Ah cool well I picked up some generics as well as the aforementioned keystone (Lowes actually carried some which surprised me). Definitely gotta snag some 3M ones though.



Edit 2:

Well it works with the official cable. I was either doing the pins incorrectly or the Cisco serial was incorrect/bad somehow? Leaving this for posterity in case it helps someone.
 
Last edited:

Aluminat

New Member
Jul 5, 2019
16
9
3
Hello all, I'm looking for managed layer 2 switch 16 port gigabit. Requirements very simple tag VLAN for wireless AP, no need PoE and can do routing local.
Currently, I have local offer 75$ for Used Allied Telesis AT-GS950/16.
Can you guys comment on this? Or recommence on used one?
 

matthew5025

New Member
Mar 21, 2016
27
22
3
definitely not, sounds like a bad optic or cabling (or host). I would get rid of the mikrotik from the path and retest to see if the issue goes away
Thanks for the reply! Further testing and I think it's the connenection between the host and the switch that has the problem.
 

eduncan911

The New James Dean
Jul 27, 2015
329
203
43
eduncan911.com
Ruckus ICX 7250 Switch Technical Specifications
^- scroll down towards the bottom

Serial port specifications (pinout mini-USB)
Pin - Signal - Description
--------------------------------------
1 - Reserved - Not used
2 - UART_RX - Receive data by Ruckus ICX 7250
3 - UART_TX - Transmit data by Ruckus ICX 7250
4 - Reserved - Not used
5 - GND - Logic ground

Serial port specifications (protocol)
Parameter - Value
----------------
Baud - 9600 bps
Data bits - 8
Parity - None
Stop bits - 1
Flow control - None


Also, created a PR against @fohdeesha 's github repo so we don't have to keep looking it up.

Add 7250/7450 Serial Pinout Details by eduncan911 · Pull Request #5 · Fohdeesha/lab-docu

:)
RE: 7250/7450 Mini-USB-B Serial Cables

So, I found this cable on eBay for HAM radio programming.

(Removed link to bad cable, don't get)

I confirmed with the seller the pinouts, which seem to match that of the Ruckus website for the 7250/7450 serial cable.

The only addition is that they supply 5V+ on pin 1 - where the Ruckus website says Pin 1 is unused.

Unfortunately, since I am remodeling, I don't have a workbench to disassemble my ICX 7250 48P to see if pin one is connected to anything.

What do y'all think? if Ruckus says Pin 1 is reserved, I'm guessing that means not in use. Thoughts?
 
Last edited:
  • Like
Reactions: tommybackeast

fohdeesha

Kaini Industries
Nov 20, 2016
2,330
2,467
113
31
fohdeesha.com
RE: 7250/7450 Mini-USB-B Serial Cables

So, I found this cable on eBay for HAM radio programming.

FTDI USB Programming Cable TH-7800 TH-8600 TH-9800 TYT Serial Mini | eBay

I confirmed with the seller the pinouts, which seem to match that of the Ruckus website for the 7250/7450 serial cable.

The only addition is that they supply 5V+ on pin 1 - where the Ruckus website says Pin 1 is unused.

Unfortunately, since I am remodeling, I don't have a workbench to disassemble my ICX 7250 48P to see if pin one is connected to anything.

What do y'all think? if Ruckus says Pin 1 is reserved, I'm guessing that means not in use. Thoughts?
I was going to say just ship me one and I'll give it a rip, but after a little digging it seems the TYT's programming port isn't RS232, it uses 3.3v TTL logic levels - (this guy says TTY but means TTL, as indicated by the TTL adapter he used, last post) TYT HT-7800 Data port pin outs

so connecting that to the ~12v logic levels on the ICX RS232 port, the 3.3v TTL signals sent to the thing won't be enough to trigger anything, and then the ICX talking back to the 3.3v TTL adapter in that cable with ~12V logic levels will probably blow out the ftdi adapter in the thing
 

Bocephus

New Member
Feb 12, 2020
9
4
3
I'm new here, and stumbled across this thread while researching an inexpensive 10gb switch options for my home lab. I'm real close to ordering two ICX6610's but before I do, I'm trying to figure out the license bit.

If I've read this thread correctly, the license is irrelevant, and can be solved?

Just wanted to clarify, because prem license price jumps from ~$200 to ~$350ish.

Thanks in advance!
 
Last edited:

ewer0012

Member
Feb 10, 2019
81
25
18
If I've read this thread correctly, the license is irrelevant, and can be solved with help from @fohdeesha?

Just wanted to clarify, because prem license price jumps from ~$200 to ~$350ish.

Thanks in advance!
Basically, yes.

If you don't mind me asking, what's the use case for two 6610s?
 

Bocephus

New Member
Feb 12, 2020
9
4
3
Basically, yes.

If you don't mind me asking, what's the use case for two 6610s?
Thanks for the quick reply.

As for the use case, my Lab currently consists of 8 physical servers, and I'm getting ready to split them between two half racks, and I wanted a top of rack switch for each rack.

I currently have 2 SG line Cisco switches, that i"m quite happy with, except for the fact they are only 1GB. For 10GB I'm using a MicroTik CRS309, and I really am not a fan.

As for all the 10GB ports, currently on my CRS309 they are used strictly for storage replication for my 3 node cluster. (Previously Hyper-V/S2D, currently running Proxmox/Ceph). I'd also like to add 10GB to a few other boxes as well over time.

Kind of long winded, but hopefully that answer the question.
 

ewer0012

Member
Feb 10, 2019
81
25
18
Why split between two half-racks? If you only have 8 servers, that's not too much space unless they're all 3u-4u. One 6610 would have 16x 10Gbe if you use the two breakout ports on the back. Would be enough for 2 connections per server, then you could utilize one or both of the QSFP ports for your storage.

The 6610s are really nice for the 2 breakout ports, giving you 16x 10Gbe ports total. I'm using the QSFP ports for my access to NAS and iSCSI respectively. Works well.

One thing I would caution about buying two (unless your power bill isn't a big deal) is that you're probably looking at about $30-$35+ for 30 days of running 2 of them 24/7. That's about what it's costing me now for 2x 48P that are mostly populated.
 

Bocephus

New Member
Feb 12, 2020
9
4
3
Most of my servers are actually desktops. I have a PowerEdge R420, 3 Precision T7600's, a Precision T5400, and the rest are re-purposed gaming rigs. So with the exception of the R420, everything is 3-4U.

As for two TOR switches, versus just the one, I was wanting to minimize the cables running between the two racks. Port count I'm sure I'd be fine with the ICX6610-48.
 

eduncan911

The New James Dean
Jul 27, 2015
329
203
43
eduncan911.com
Most of my servers are actually desktops. I have a PowerEdge R420, 3 Precision T7600's, a Precision T5400, and the rest are re-purposed gaming rigs. So with the exception of the R420, everything is 3-4U.

As for two TOR switches, versus just the one, I was wanting to minimize the cables running between the two racks. Port count I'm sure I'd be fine with the ICX6610-48.
^- this person is a clone of me. A real "home lab" user repurposing things to learn, that can't afford the pretty $10,000 setups in r/homelab. Mine were old mining and gaming rigs myself. Though I've since downsized, got rid of most, but let only a single purpose Godzilla server running 5 VMs plus K8s for a dozen Tor and NZB docker images mixed with a dozen UP Boards as part of the K8s cluster.

Only real non-k8s machine running is my UP board router that supports AES-NI for my openvpn, which is getting replaced soon with Frankenstein Qubes-like router I've been tinkering with soon for better isolation.

I just wished AMD had IoT solutions like these UP boards, as Ive gotten rid of most Intel from my household.
 

Bocephus

New Member
Feb 12, 2020
9
4
3
^- this person is a clone of me. A real "home lab" user repurposing things to learn, that can't afford the pretty $10,000 setups in r/homelab. Mine were old mining and gaming rigs myself. Though I've since downsized, got rid of most, but let only a single purpose Godzilla server running 5 VMs plus K8s for a dozen Tor and NZB docker images mixed with a dozen UP Boards as part of the K8s cluster.

Only real non-k8s machine running is my UP board router that supports AES-NI for my openvpn, which is getting replaced soon with Frankenstein Qubes-like router I've been tinkering with soon for better isolation.

I just wished AMD had IoT solutions like these UP boards, as Ive gotten rid of most Intel from my household.
Yeah, my lab keeps getting bigger, and so does the electric bill. I often wonder if it wouldn't be cheaper in the long term to pick up ~5 R630's and get rid of my 7 Desktops/Servers.
 

infoMatt

Active Member
Apr 16, 2019
220
99
28
Yeah, my lab keeps getting bigger, and so does the electric bill. I often wonder if it wouldn't be cheaper in the long term to pick up ~5 R630's and get rid of my 7 Desktops/Servers.
From a pure energetic standpoint, nothing beats consolidation and virtualization: it's better to have a pretty big machine with lots of idling cores and RAM than to have half a dozen or more of smaller servers, all of them underutilized. A "modern" dual socket server can idle below 75ish W, and it can perform better than 3 or 4 "desktop" PCs.
The break-even point however will depend of course on the price of the components that you can find vs. the increased power bills.
 

eduncan911

The New James Dean
Jul 27, 2015
329
203
43
eduncan911.com
Yeah, my lab keeps getting bigger, and so does the electric bill. I often wonder if it wouldn't be cheaper in the long term to pick up ~5 R630's and get rid of my 7 Desktops/Servers.
To echo @infoMatt, this was one of the primary reasons I consolidated into a single BFM (Big Fracking Machine). All of the machines I had running was something like 400 to 500W idle, and a LOT of heat in the closet. I'm on my third build of the BFM right now and finally got the power levels decent (~75W).

The two other primary reasons was heat generation and exploit/attack surface (I'm in CyberSec, and just can't stand to run unsecure hardware/ports no matter the "low threat model".

Previously, I build an Intel 2690V3 (12C) with an engineering sample and 128 GB of ECC DDR4 Buffered ram. Idled with 14 HDDs at 75W due to the 9W LSI card and 12W Supermicro SC846 chassis backplane. It wasn't enough cores.

Then I went with dual 2640LV4s (14C each, 28C/56T total) ES in a Supermicro board with no IPMI (to remove those CVEs from my scans, rigged up a RPi with serial console instead to power on/off remotely on an airgapped VLAN). Got the CPUs cheap on ebay since they were ES. Same 128 GB DDR4 ECC Buffered ram. Idle was now 81W, due to the chipset and dual CPUs, along with a Q2000 nvidia card for hardware encoding/decoding of movie streams. If I removed a single CPU, I could get it down to around 70W idle. Which led me to my collection of SBC UP Boards to now handle most of my home infra and apps needs.

Finally, I got sick of the Intel exploits, Intel ME disable from each BIOS update, etc... And went all in with AMD.

Current server is AMD 2950X (16C, 32T), more than double the Mhz of the engineering sample versions of the Intel chips above (and more than double, sometimes 3x the CPU Mark ratings per core!). And, I replaced my Port Expander backplane in the SC846 chassis with a JBOD version (saving 12W, 24 SATA hookups now)), dumped the LSI card (~9W saving) for 10 onboard SATA ports along with replacing the 12x 4TB drives with 8x 10TB and 12TB drives (this was the most expensive upgrade of them all - I've done it piece meal over the last two years, slowly...). And, I had to switch to Unbuffered ECC 64 GB DDR4 ram as the AMD X399 chipset only supports Unbuffered, not the Buffered ECC I had in the Intel boards. :( But i did notice I wasn't using that much ram any longer, with all the UP Boards I have running in clusters.

I'm now down to ~75W idle with that beast. AMD Threadripper and X399 isn't as power efficient at idle as the Intel chipsets. In a few years, I may downscale to the Ryzen consumer models with 16C as this was kinda overkill, and the newer 3000 series is just about as powerful - with much lower TDP. The UP Boards, per 4x board cluster of the UP Squared Pentium versions hovers around ~8W when idle. But they are hardly idle, always processing some NZB, Tor relay and services, and constant metric/logs/SNMP collections. All in all, I'd say per cluster averages around 15 to 20W per 24 hours - remember, this is running like two dozen docker images and services, log collections and parsing, metric compaction nightly, etc - they get heavy heavy use. I have two clusters I switch between as I am constantly tearing down and rebuilding the k8s clusters in my homelab for work. I have a pretty good immutable setup now using PXE boot images created from Packer - so I can easily replace a single bare metal UP board at any time (I really need to blog this setup).

I run the latest Xen on ArchLinux on everything for the latest CVE fixes.

Which lead me now to this thread about Brocade switches... I need to get a handle on my VLANs and get some POEs for new security cameras, as well as lowering even more wattage. I'm now building up my single ICX 7250 48P for that task, thanks to this thread, to replace a number of POE, managed/unmanaged, and dumb switches.
 
Last edited:

matthew5025

New Member
Mar 21, 2016
27
22
3
Thanks for the reply! Further testing and I think it's the connenection between the host and the switch that has the problem.
Just placing this here to help anyone who might run into the same issue:

The switch gave itself an IP address from DHCP and assigned it to the uplink port, and effectively did not forward ARP messages, causing the issue. Running no ip address solved the issue