Brocade ICX6450 / ICX6610 / ETC

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. AlphaG

    AlphaG Member

    Joined:
    Jun 8, 2017
    Messages:
    70
    Likes Received:
    14
    I have a general network configuration question and have three 6450 switches in different locations because some parts of the house are old and others newer.

    Currently internet comes into the house via cable company into a first floor closet. This is nowhere near the rest of the equipment so the open internet WAN connection travels up two stories to the attic above second floor, across 30 to 40 feet, then down two stories to a packed equipment cabinet that has mostly AV and automation gear. My pfsense box is in here. The LAN starts here with feed to the cabinet it’s in, and branching to other consumer switches via cat5e wires going to a second story new construction closet where my ‘home lab’ is, as well as back to the original closet where the cable came in to another small consumer switch that supplies the office computer next door to it.

    The new setup has three 6450’s stacked, one in each of these locations, as soon as I run the fiber.

    Can I segment off a vlan, say vlan999, for the WAN connection and eliminate my long cat5e run from the equipment cabinet to the router? Do I need to assign a vlan, say vlan10 to the remaining ports on the stacked switches if I do so? Or am I just asking for trouble?
     
    #1201
    tommybackeast likes this.
  2. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    yeah pretty easily, I do the same at home. I would make normal traffic vlan 10 or something like you said, untagged on all ports, tagged on the ports between switches. Same with WAN, untagged on the port the ISP connects to, tagged on the ports between switches. That way the inter-switch links are only running tagged traffic and you don't have to deal with dual-mode. Just be sure to *not* put a VE or any other type of interface in the WAN vlan, or the ISP will see it as customer equipment instead of a passive link to your actual CPE

    here's a snip of what mine looks like:


    Code:
    vlan 5 name WAN by port
     tagged ethe 1/3/8 ethe 2/3/8
     untagged ethe 1/1/1
    
    
    vlan 10 name LAN by port
     tagged ethe 1/2/6 ethe 1/3/8 ethe 2/2/6 ethe 2/3/8
     untagged ethe 1/1/2 to 1/1/48 ethe 1/2/7 to 1/2/10 ethe 1/3/1 to 1/3/7 ethe 2/1/1 to 2/1/48 ethe 2/2/7 to 2/2/10 ethe 2/3/1 to 2/3/7
     router-interface ve 10
     
    #1202
    AlphaG and tommybackeast like this.
  3. maes

    maes New Member

    Joined:
    Nov 11, 2018
    Messages:
    21
    Likes Received:
    11
    I only have a 6450-24, non-POE. By nature it would generate a LOT less heat, but it also seems the fan voltage doesn't drop as much on the -24 model. The Noctuas never completely stop spinning in 'low speed' mode. However, they move so little air around that the temperatures start creeping up pretty quickly.

    I will likely mod them to be permanently powered off the 12V line instead of letting the board control their speed. Even at full speed they're basically silent, so there's no point in running them slower.

    I don't know what the 'default' fan configuration is on the -48p, but 3 noctuas are probably too little if it uses 2 fans normally. 2 of the noctuas together move just about as much air as the single 'factory' 40mm fan does on the -24.
     
    #1203
  4. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    default fans on the 48P are 3, but with no PoE load it draws the same amount of power and puts out the same amount of heat as the 48 non-poe, increasing a few watts per PoE device depending. the three fans are sized to cool 770 watts of PoE load. if you have 4 or 5 2-watt PoE cameras you could run one stock fan, I have been for some time. These things don't mind running warm, the stock software doesn't even bother spinning the fans faster until 70 or 75c. 50c is nothing
     
    #1204
    Last edited: Jan 12, 2019
  5. Snorf

    Snorf New Member

    Joined:
    Nov 12, 2018
    Messages:
    9
    Likes Received:
    5
    Need a little help, sorry if this has been answered but I cannot find confirmation. I just want to make sure before ordering a bunch of stuff that may end up as useless.

    Background: I have 2 – ICX6610-48P’s, and I have some HP 656089-001 Infiniband 10/40GB DP NIC 649281-B21 cards to be used in my server(s) and computer(s) at 10gbe and 40gbe.

    I am not going to stack the 6610’s yet, but probably in the future.

    Since I am not stacking yet I wanted to use the 40gbe ports on the 6610 for servers and comps. I have read the stacking useful hints and I understand I need Type B cables or Type A and flip the end as in the video with the paper clip to stack (all cables female x female). But……..(some FS links for clarity only)

    Questions:

    1. For 40gbe from ports 1/2/1 and 1/2/6 do you use Type A or Type B MTP/MPO cables plus transceivers to hook the 6610 to my NICs in a server/comp?

    2. Would these cables work for stacking, and for 6610 to server/comp at 40gbe?

    2.1. Mellanox MC2206130-001 40G QSFP+ DAC Cable

    2.2. 1m (3ft) Mellanox MC2206310-001 Compatible 40G QSFP+ Active Optical Cable

    3. Does it matter if you use 8 fiber or 12 fiber MTP/MPO cables to make these connections, 6610 to NIC, or for stacking?

    4. If using an MTP/MPO to LC cassette does it matter about 8 fiber or 12 fiber MTP/MPO cables from 6610? I understand that the Cassette will say if it is flipped/Type A/B for hook-up. But in post #5 of this thread I am a little confused by the 24 port patch panel. It only has 4 MTP/MPO connections but breaks out to 24 SFP+ (12 fiber cable). I only have 16 ports of 10gbe available, do 8 ports just not work? Do I need to worry about 8 fiber vs 12 fiber MTP/MPO cables and how does it effect how/where I use them?

    Thanks in advance for clearing this up for me.

    Snorf
     
    #1205
  6. kapone

    kapone Active Member

    Joined:
    May 23, 2015
    Messages:
    522
    Likes Received:
    181
    #1206
    Snorf likes this.
  7. ljvb

    ljvb Member

    Joined:
    Nov 8, 2015
    Messages:
    76
    Likes Received:
    7
    Quick question.. might beed to post this in the VMWare section, but I'll check here first. What is the hashing algorithm for VMware for the 6610 LAGs. Trunk type shows hashed based, but I have a ton of options when setting up the vDS, normally I picked source and destination based IP. Hopefully that is right.

    One of my lags interfaces is showing inactive.
    Port [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
    1/2/2 1 1 20001 Yes L Agg Syn Col Dis Def No Ina
    1/2/3 1 1 20001 Yes L Agg Syn Col Dis No No Ope

    The second lag shows both operational. Side note, this interface is also not showing up in VMWare (that is off topic here, but possibly related, I am going to post that in the VMWare section as I am having other problems with a second server).

    Related to that server.. for some reason both 1/2/4 and 1/2/5 were showing down on both the switch and VMWare. I could not for the life of me figure out why, but pulling the power from the switch and restarting seems to have corrected that, but they keep going down after a few days. I am using the Avago 40/10 breakout cable linked much further up in the thread. Not sure what is happening. I had no issues when I used just two of the ports, now that I am using all 4, things have been going haywire. Now there is apparently a known issue with the intel X520-DA2 and the vsphere 6.7 update.. but the links remain down even when power cycling the server, they should at least show a link....

    Dunno.. might have to dump lacp and just use 1 10G link in each box and a bunch of 1G links.
     
    #1207
  8. Joao Paulo Mello

    Joao Paulo Mello New Member

    Joined:
    Jan 16, 2019
    Messages:
    8
    Likes Received:
    1
    Hi guys, it seems that I'm bit late for the party. Comparing ICX64xx prices (at ebay) from the first post to this one that seems crazy, you people definitely inflated the market and for me is even harder cos I have to pay $150 more for the USPS shipping if I got lucky.
    But better late than never. I'm literally hunting for a ICX64xx-P. Personally I prefer a quiet 6450-P but 8x10GB SFP+ on the front of a 6610 is tempting. But I know noise. I'm a proud owner of ASA-5512x and a Nexus 3048 and I had to mod both to keep my life going forward without going insane. We all know, the place where we live aka home isn't a place those devices were designed to work. So we complain, whinings here and there but it is what it is. What we can do is to avoid these gears but I don't know. I hate them but I love them, so I try to acommodate things the way I can keep it bearable, people stash everything away on the basement like we all do with hammers and pliers to keep the heat and noise away. I can't keep my gear away from me. Most of my life I spent coding for a living systems that supports network infrastructure, so I have to interact with the equipment all the time. Currently I'm not actively working everyday, just decided to slow down a bit and spend some pings with the wife. I have now mostly Cisco gears, which is the last company I worked for and I still work sometimes as a consultant. The thing is, I want to install surveillance gear around the house and I'll get rid of my Nexus 3048TP and get a POE switch. I have 2 ESXi hosts and a NAS using the 4 SFP+ ports from the Nexus for storage. I need Layer3 at switch level to run simulations and other stuff. I read somewhere the 6450 doesn't support VRF-Lite, is that true?
    I read some posts where people says 6610 is very noisy during initialization but the fans slow down after that just like most network gear. The ASA 5512x does the same but nothing near bearable, so I modded the fans using Noctua reducer cables, isn't dead silent but it's ok, the ASA doesn't complain and just get 4C/6C hotter.
    So I'm thinking which one, risk everything for the looks and 16x10Gbe (6610) or keep it safe by choosing the safe heaven (6450).
    before I go, thanks everyone for this thread specially to @fohdeesha. I'm always on STH, great articles and everything but when I read this thread I had to sign up.
    Sorry by the mistakes I've made with the language. I lived for some years in US but it was long time ago.
    Be safe.
     
    #1208
    mathiastro likes this.
  9. DanielWood

    DanielWood New Member

    Joined:
    Sep 14, 2018
    Messages:
    26
    Likes Received:
    9
    The 6610 never gets quiet, and there is not yet a way to silence it (replacing fans wont work because it expects minimum RPMs and fails to boot). If noise is a concern, avoid the 6610.
     
    #1209
  10. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757
    the 6610 never gets silent/very quiet, but with rev B power supplies it should spin WAY down after boot, and you shouldn't be able to hear it behind a closet door. I suspect the people with very loud 6610's in this thread have rev A PSUs
     
    #1210
  11. Joao Paulo Mello

    Joao Paulo Mello New Member

    Joined:
    Jan 16, 2019
    Messages:
    8
    Likes Received:
    1
    Rev B Dual PSU's right? Is there any way to predict the PSU's revision just by serial numbers or something? I mean, things that we can ask for the seller without the need to open the equipment? I'm willing to take a shot on 6610 with Rev B PSU's.
     
    #1211
  12. frameshift18

    frameshift18 New Member

    Joined:
    Jan 9, 2019
    Messages:
    4
    Likes Received:
    0
    This thread has convinced me to pick one of these up! I was concerned they would be extremely loud but it's actually a lot quieter than I would have thought based on some feedback on here. I do have a single rev c and the exhaust air direction.
     
    #1212
  13. frameshift18

    frameshift18 New Member

    Joined:
    Jan 9, 2019
    Messages:
    4
    Likes Received:
    0
    The revision number is printed on the sticker on the psu, all they'd have to do is unscrew 2 screws and pop it out real quick. It's not like they have to boot it up and run commands on the CLI to check or anything.
     
    #1213
  14. Joao Paulo Mello

    Joao Paulo Mello New Member

    Joined:
    Jan 16, 2019
    Messages:
    8
    Likes Received:
    1
    Thanks @frameshift18. That's good to know I'll ask them and see how it goes.
    Guys, there's this listing on ebay. 6610-P dual psu and it seems with 10G license installed, but too expensive IMO, even with free shipping to my country.
    Brocade ICX 6610-48P ICX6610-48P-E 48-Port Gigabit PoE+ Managed Network Switch | eBay

    Seller has 9 units. I hope prices starts coming down a little very soon.

    Edit: @fohdeesha .
    Can you please keep a 10G license for the 6610 hidden somewhere? I'll need one and hopefully very soon.
     
    #1214
    Last edited: Jan 17, 2019
  15. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,045
    Likes Received:
    757

    [​IMG]
     
    #1215
  16. ljvb

    ljvb Member

    Joined:
    Nov 8, 2015
    Messages:
    76
    Likes Received:
    7
    I figured out my problem.. I think.. It appears to be the breakout cable.. something funky is going on.. at least I hope it is the cable. I picked up a Cisco 40x10 breakout copper one which should arrive tomorrow. Guess I'll figure out whether it was the cable or not.
     
    #1216
  17. juey

    juey New Member

    Joined:
    Oct 1, 2018
    Messages:
    20
    Likes Received:
    3
    Must be, i have no problems at all with my 6610 connected to x520 using vsphere 6.7. Im using the breakout from FS.
     
    #1217
  18. ljvb

    ljvb Member

    Joined:
    Nov 8, 2015
    Messages:
    76
    Likes Received:
    7
    #1218
  19. Joao Paulo Mello

    Joao Paulo Mello New Member

    Joined:
    Jan 16, 2019
    Messages:
    8
    Likes Received:
    1
    I know this was answered before but I couldn't find it. I'm almost pulling the trigger on that overpriced 6610-48p. The switch has 2 PSU's, that's great there's a chance fans will slow down after initialization ( the nexus 3048TP has the same behaviour. A banshee with one PSU, a smaller banshee with 2 ). The listed switch has only one fan cage. 2 fan modules helps in some way this picture? Maybe slow down the PSU's fan a little more?

    Edit: The 6610-24p is quieter?
     
    #1219
    Last edited: Jan 18, 2019
  20. ljvb

    ljvb Member

    Joined:
    Nov 8, 2015
    Messages:
    76
    Likes Received:
    7
    You all need to quit complaining about the noise.. You don't know noise till you have a couple of these in your garage rack.... The fans on the C3000 blade chassis are Boeings on take off.. all the time....

    [​IMG]
     
    #1220
    Joao Paulo Mello likes this.
Similar Threads: Brocade ICX6450
Forum Title Date
Networking Brocade ICX6450 - Help with VLAN Config/SFP Licensing Question Oct 3, 2018
Networking Brocade 6650 Mar 9, 2019
Networking Brocade ICX7400-1X40GQ vs ICX7000-1X40GQ Feb 13, 2019
Networking Brocade ADX 1000 Software Jan 31, 2019
Networking Brocade VDX 6740 Jan 18, 2019

Share This Page