Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

dlaffin1012

New Member
Feb 21, 2019
12
0
1
I also enjoy building these systems. But at a $300 price tag and extra NICs, I don't know if it is worth the cost.
 

Zervun

Member
Feb 2, 2019
44
7
8
Oregon
I also enjoy building these systems. But at a $300 price tag and extra NICs, I don't know if it is worth the cost.
I think you are probably boiling the ocean. Like Kapone says it's up to you what you want to do with it. If you desire ultimate redundancy then you can stack and split ports across the two and/or play around with LACP and the like.

You can also always expand later. The odds of these enterprise grade switches toasting is probably fairly low and they can be fitted with dual power supplies. You mentioned homelab vs. home. I think probably for the majority of us it is one and the same or some mix of the two. I run all my servers, NAS, esxi, etc off of the 6610 but it also servs as the backbone for my home.

I bought 2x 6610's for more 40g ports, but ended up simplifying and just using 10g on some of the NASs that I was going to put on 40g so i wouldn't have to use 2 6610s and in turn saving on some power and noise. My other 6610 is powered off waiting to swap in if need be but I have no expectation that the primary is going to go out. I've got 42u filled with stuff so any little bit of savings on power/noise I try to take.

I've had my internet go out way more often than my internal network which puts me in a much larger predicament. Although my wife gets grumbly when the SageTV goes down ;).
 

SuperServer

New Member
Feb 1, 2019
15
1
3
Ok so i am not understanding what is going on with vlan set up. Default vlan 1 is obviously working no issues.

I create vlan 10 for instance i move my server and main pc to it. Vlan 10 will not work but everything else in vlan 1 is ok.
I create vlan 10 again i move my server, PC, AND port 1/1/1 (router port) i get vlan 10 working. BUT i can not acccess the switch AND vlan 1 does not work.

This is my prevlan show run config. What am i doing wrong in the set up?


Code:
Current configuration:
!
ver 08.0.30saT7f3
!
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
 router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
enable telnet authentication
hostname route
ip dhcp-client disable
ip route 192.168.2.0/24 ethernet 1/1/1
ip add-host-route-first
!
username intertubes password .....
!
!
clock summer-time
clock timezone gmt GMT-05
!
!
ntp
 disable serve
 server 216.239.35.0
 server 216.239.35.4
!
!
!
!
!
!
!
!
!
interface ethernet 1/2/6
 tag-profile enable
!
interface ethernet 1/3/1
 speed-duplex 10G-full
!
interface ethernet 1/3/2
 speed-duplex 10G-full
!
interface ethernet 1/3/3
 speed-duplex 10G-full
!
interface ethernet 1/3/4
 speed-duplex 10G-full
!
interface ethernet 1/3/5
 speed-duplex 10G-full
!
interface ethernet 1/3/6
 speed-duplex 10G-full
!
interface ethernet 1/3/7
 speed-duplex 10G-full
!
interface ethernet 1/3/8
 speed-duplex 10G-full
!
interface ve 1
 ip address 192.168.2.2 255.255.255.0
!
arp 192.168.2.3 0002.c942.85d0 ethernet 1/2/6
!
!
!
!
!
!
!
!
!
end
 
Last edited:

dlaffin1012

New Member
Feb 21, 2019
12
0
1
Alright Zevrun, you helped clarify a bit more explaining how much trust you have in the networking in your own home. Thats a big thing for me, to see how much trust or knowledge from other's own experiences. I will probably do the same thing and have one 6610-48p and run everything off that and at some point have a second as a cold spare. Then I can get single port cards and reuse some that I am currently using. I have enough running that saving energy is up there on the priority list, not the top but close. I appreciate the help in this thread (let alone the entire forum) as it is really nice to see the experiences and knowledge brought together about stuff like this. I will look for a deal on a 6610-48p at some point. I am still gonna get one with dual psu's though.
 

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
656
234
43
Hey, quick question that I couldn't manage to Google. Is there any way to silence the console spam about fan status? I've got a Pi I'm using as a console server attached to one of my 6450s and reconnecting to my screen session basically looks like this:

Code:
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
I'm not sure if this is normal or a result of replacing the fan with the quieter model I bought a few months ago.
 

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
656
234
43
Ok so i am not understanding what is going on with vlan set up. Default vlan 1 is obviously working no issues.

I create vlan 10 for instance i move my server and main pc to it. Vlan 10 will not work but everything else in vlan 1 is ok.
I create vlan 10 again i move my server, PC, AND port 1/1/1 (router port) i get vlan 10 working. BUT i can not acccess the switch AND vlan 1 does not work.

This is my prevlan show run config. What am i doing wrong in the set up?
You need some way to route so that different vlans can reach each other. You could use ve's on the switch or use an external router. I use an external firewall (pfSense in router on a stick configuration) to route my traffic so I can control traffic between my different networks/vlans.

This should help:
 
Last edited:

SuperServer

New Member
Feb 1, 2019
15
1
3
I did that in one of the configurations a few times following the set up and I still couldn’t get it.

Modem x.x.2.1
Switch x.x.2.2
Vlan10 x.x.10.1
I created a static route vlan10 to modem and I couldn’t get back to it.

If you are routing through layer 3 in the switch what is the static route set to?

If port 1/1/1 is my router... should it be set in vlan10 and in default vlan1? Set in both as tagged?
 

Kewjoe

New Member
Feb 3, 2019
6
2
3
Hey, quick question that I couldn't manage to Google. Is there any way to silence the console spam about fan status? I've got a Pi I'm using as a console server attached to one of my 6450s and reconnecting to my screen session basically looks like this:

Code:
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
Rear Side Fan 1  is down
Rear Side Fan 1  is up
I'm not sure if this is normal or a result of replacing the fan with the quieter model I bought a few months ago.
Which fan did you end up with?
 

SuperServer

New Member
Feb 1, 2019
15
1
3
You need some way to route so that different vlans can reach each other. You could use ve's on the switch or use an external router. I use an external firewall (pfSense in router on a stick configuration) to route my traffic so I can control traffic between my different networks/vlans.

This should help:
I have done that quite a few times. Through CLI and the web interface. Every time i do it vlan10 never comes up with internet. Unless i have to write mem and reload? I dont save it because i can always reboot back into the original configuration if something isn't correct.

Modem 190.1.2.1
Switch 190.1.2.2
Vlan10 ve 10.10.10.1
Assign ports 1/2/6 and 1/2/1 (40g pc and server connects)

After that i check internet vlan1 is fine but nothing in vlan10 can connect or connect to the switch. Is there a static route i am missing or something?
 

itronin

Well-Known Member
Nov 24, 2018
585
331
63
Denver, Colorado
I have done that quite a few times. Through CLI and the web interface. Every time i do it vlan10 never comes up with internet. Unless i have to write mem and reload? I dont save it because i can always reboot back into the original configuration if something isn't correct.

Modem 190.1.2.1
Switch 190.1.2.2
Vlan10 ve 10.10.10.1
Assign ports 1/2/6 and 1/2/1 (40g pc and server connects)

After that i check internet vlan1 is fine but nothing in vlan10 can connect or connect to the switch. Is there a static route i am missing or something?
yes. assuming your posted config is still accurate and I didn't miss it (its late and my eyes are tired) - I did not see a default route or route for 0.0.0.0 via your firewall (modem?) in your config. that will need to be in your switch unless you are also trying to configure a routing protocol to pass a default route - which again i did not see. Your hosts in vlan 10 will need to know that their default gateway which is the switch's interface for vlan 10.

you mentioned setting up a vlan 10 route somewhere. You will need a route on your modem to vlan 10 which will be via the switch's vlan 1 interface... so on the modem something along the lines of
network 10.10.10.0 netmask 255.255.255.0 gateway 190.1.2.2 metric <yourvalue>
also is your ve 1 interface 190.1.2.2 or 192.168.2.2? your config shows one thing your last post shows another.

Best practice would be to configure a /30 network between the firewall (modem?) and the switch but if the modem is a vendor cable modem or dsl modem or somesuch and you are also using it for dhcp and maybe dns(?) on vlan 1 then you probably don't want to do that - besides work on getting basic subnet to subnet routing working.

also depending on your modem you may need to tell it something about NAT in order for it to properly NAT traffic coming from vlan 10.


itr
 
Last edited:

goxy

New Member
Mar 4, 2019
3
0
1
Guys, I'm also in.

I found you 2 hours ago, and just ordered 2xBrocade ICX6610-24-I Switches from ebay for 818$ with shipping(I hope will not receive rocks)
Brocade / Ruckus ICX6610-24-I-BRC - 24 port 1G RJ45, plus 8 x 1G SFPP uplinks ports (upgradeable to 10G). 4 x 40G stacking ports. Intake air flow. Base S/W
Complete new/un-used. New sealed original box.
BTW. Last time I touch enterprise equipment was 8 years ago. In that time I worked for Cisco silver partner,.. I got CCNP, CCVP and I think a wireless. I started to prepare CCIE for Voice with ESXi, used Qemu to virtualize IOS and all demo lab......
 

Zervun

Member
Feb 2, 2019
44
7
8
Oregon
Guys, I'm also in.

I found you 2 hours ago, and just ordered 2xBrocade ICX6610-24-I Switches from ebay for 818$ with shipping(I hope will not receive rocks)
Keep in mind those are reverse fan flow - the I means the exhaust flow goes from back to front - the E models go front to back. Can of course place them reverse so the ports are facing the back of a cage.
 

goxy

New Member
Mar 4, 2019
3
0
1
Hi Zervun, thanks for explanation, so I have to just reverse fans in the case that I need flow from front to back.

Do you know what means from description " 8 x 1G SFPP uplinks ports (upgradeable to 10G)", does I have to replace come cards if I need upgrade to 10GBs ports?
 

Zervun

Member
Feb 2, 2019
44
7
8
Oregon
Hi Zervun, thanks for explanation, so I have to just reverse fans in the case that I need flow from front to back.

Do you know what means from description " 8 x 1G SFPP uplinks ports (upgradeable to 10G)", does I have to replace come cards if I need upgrade to 10GBs ports?
Not sure if you can reverse the fans easily. They come in modules and the power supplies which you would have to open up (AFAIK). You might have to just mount them to the back of a cage and turn them around if you desire the exhaust flow direction.

You have to have licenses for enabling the 10g ports, there are no cards that need to be put in, just licensing. See the first page of the thread for information on how to get the licenses.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,084
1,943
113
30
fohdeesha.com
Guys, I'm also in.

I found you 2 hours ago, and just ordered 2xBrocade ICX6610-24-I Switches from ebay for 818$ with shipping(I hope will not receive rocks)


BTW. Last time I touch enterprise equipment was 8 years ago. In that time I worked for Cisco silver partner,.. I got CCNP, CCVP and I think a wireless. I started to prepare CCIE for Voice with ESXi, used Qemu to virtualize IOS and all demo lab......
$818!?!?! holy crap - what country are you in?
 

goxy

New Member
Mar 4, 2019
3
0
1
Not sure if you can reverse the fans easily. They come in modules and the power supplies which you would have to open up (AFAIK). You might have to just mount them to the back of a cage and turn them around if you desire the exhaust flow direction.
Yes, I will check when equipment arrive, thanks.
You have to have licenses for enabling the 10g ports, there are no cards that need to be put in, just licensing. See the first page of the thread for information on how to get the licenses.
Great, so just software limitation.

$818!?!?! holy crap - what country are you in?
I'm from Serbia, 2000km from Denmark(Europe) where seller is located.
The price for 2 new/un-used with shipping, I think that is excellent price for Europe.
 

ViciousXUSMC

Active Member
Nov 27, 2016
201
100
28
38
Worked on this yesterday as my fans came in.
Goes over the fan mod, things I had to do, things to look for, and of course the net gains in noise reduction and temperature changes.


Have a ICX 6450-24P on the way and will do this again with that one if its as loud as my 48P was.
 

Blue)(Fusion

Member
Mar 1, 2017
86
16
8
Has anyone seen something similar to this or know what might be causing the issue?

I currently don't have any POE devices to test with, probably will order something fast I guess, but wanted to know if this is normal.

The switch just arrived yesterday and I've went through the entire initial setup and flashing to current firmware.

Code:
PoE: Stack unit 1 Power supply 2  with 748000 mwatts capacity is up
PoE Info: Adding new 55V capacity of 748000 mW, total capacity is 1496000, total free capacity is 1496000
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
PoE Event Trace Log Buffer for 2000 log entries allocated
PoE Event Trace Logging enabled...
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
PoE Info: Hard Resetting in slot 1....
PoE Info: Resetting module in slot 1....completed.
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
I just got my 6450-24P today and low and behold, I have the same issue. Booooo.