Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[fohdeesha]

can't stack the 6450 with a 6430

 

[Bell System]

About g-electronics.

I think the initial post from Hakujou was offset with the same experience with another seller's 6610. rgunner obtained a full refund. Most likely they obtained some bad switches that had damaged PoE modules.

My 6610-48P came from them and I have to state that it works fine for the lab. All ports work with PoE. All SFP+/QSFP+ ports work with the Mellanox DACs I have access to. The only issue is one of the link LEDs doesn't work. Prior to buying I asked them what PSU versions they had on hand and whether they had ears because their photos didn't show any. They allocated one with Rev B PSU along with with ears. Their customer support is pretty awesome, considering what rgunner experienced.

Yeah, I'm sure they'll make it right if there are any troubles.

can't stack the 6450 with a 6430

Thanks for confirming my suspicion.

 

[Hakujou]

About broken 6610s... I finally decided myself to order a third one from a third seller. HOPEFULLY this time it won't have the issue I had with the first ones.
I also sent a message to my second seller (fcmanagement), I'll let you know how this goes...

I really hope the third will be the last because I'm starting to getting tired honestly. If the third one does not work I'll probably buy a 7250...

EDIT: I have a dumb question. Does a PoE PSU on a non-PoE unit enables it ? I assume not, it probably has some additional controllers I guess.
And if not, would a PoE PSU work on a non-PoE Unit (even if it does not enable it, will it work ?)

 

[kapone]

The PoE power supplies are essentially two discrete power supplies in a single unit.

If you plug a PoE power supply into a unit, that does not have PoE functionality/innards - it'll work as usual, with no PoE functionality.
If you plug a PoE power supply into a unit, that does have PoE functionality/innards - it'll work as usual, with PoE functionality.

If you plug a non-PoE power supply into a unit, that does have PoE functionality/innards - it'll work as usual, WITHOUT PoE functionality.
If you plug a non-PoE power supply into a unit, that does not have PoE functionality/innards - it'll work as usual, well...duh.

 

[cuco]

hey,
my 6610 finally arrived. so powered up and just wanted to login to the webintereface. What is default user and password? tried admin/password but this did not work. thanks for the help

 

[fohdeesha]

hey,
my 6610 finally arrived. so powered up and just wanted to login to the webintereface. What is default user and password? tried admin/password but this did not work. thanks for the help

the update + config guide in big bold letters in the original thread post

 

[cuco]

Thanks. i wil have a look!

 

[Hakujou]

I have another (dumb) question. I have two "deads" ICX 6610-48P switches (ASIC errors, see my previous posts) but the PoE part was working fine.
Next week I'll receive a non PoE unit (also 48 ports). Can I transplant a PoE board and the power supplies from one of the PoE units to the non-PoE one ?

 

[fohdeesha]

it's been a long time since I've looked, but I THINK so assuming the main board has the headers for it. I know the PoE units identify themselves to the software as PoE models with some bit set in the chassis EEPROM and I don't know which one it is, so you'll need to copy the eeprom from the PoE unit to the non-PoE unit which isn't hard, just run the following in the bootloader to get the eeprom contents:

i2c read 1 0 256
i2c read 1 100 256

copy the output of both commands to a text file somewhere

 

[cuco]

Which 40gbit card would you recommend to use with the icx6610?

 

[Hakujou]

it's been a long time since I've looked, but I THINK so assuming the main board has the headers for it. I know the PoE units identify themselves to the software as PoE models with some bit set in the chassis EEPROM and I don't know which one it is, so you'll need to copy the eeprom from the PoE unit to the non-PoE unit which isn't hard, just run the following in the bootloader to get the eeprom contents:

i2c read 1 0 256
i2c read 1 100 256

copy the output of both commands to a text file somewhere

Should I run that on both PoE and non-PoE units and send them to you to compare ? I'm not sure of what to do.

Which 40gbit card would you recommend to use with the icx6610?

Mellanox ones are pretty popular around here. They're cheap and they work fine.

 

[cuco]

Is this a good matching one?
STANDARD PROFILE Mellanox ConnectX-3 VPI Dual-Port 2x 40GB IB QSFP+ PCIe x8 3.0 | eBay

Or is this pro version a better choice? What's the difference?

 

[Hakujou]

Is this a good matching one?
STANDARD PROFILE Mellanox ConnectX-3 VPI Dual-Port 2x 40GB IB QSFP+ PCIe x8 3.0 | eBay

Or is this pro version a better choice? What's the difference?

Pro has RoCE v2 and some offloading for vxlan I think.
This card will be just fine if you don't need those.

 

[fohdeesha]

Should I run that on both PoE and non-PoE units and send them to you to compare ? I'm not sure of what to do.

sure, ill send back the commands to write over on the PoE model. to ensure we're not wasting our time, before doing anything I would open up the non poe and ensure it has the appropriate headers etc

 

[fohdeesha]

Is this a good matching one?
STANDARD PROFILE Mellanox ConnectX-3 VPI Dual-Port 2x 40GB IB QSFP+ PCIe x8 3.0 | eBay

Or is this pro version a better choice? What's the difference?

buy a $25 649281-B21 on ebay and use this post to convert it to regular mellanox: https://forums.servethehome.com/ind...net-dual-port-qsfp-adapter.20525/#post-198015

 

[Hakujou]

sure, ill send back the commands to write over on the PoE model. to ensure we're not wasting our time, before doing anything I would open up the non poe and ensure it has the appropriate headers etc

Thanks a lot for your help.
Looks like the switch hasn't left the USA yet, so I'm not expecting to get it before another 1-2 weeks. I'll send them when I got it.

 

[spali]

So what I am going to do with mine is stack across the breakout ports, and use the 4 total ports across my 2x 6610's for 40g.

Does this also work if I use the 40g's as uplink to a 40g switch?
My plan is to uplink two 6610 to my arista 40 g "core" switch.
But if I understand it right, this forces me to not stack or stack with the breakout ports. I wonder how I could force the main traffic to the core switch over the 40g uplinks in a "breakout" stack.
Any ideas or suggestions?

 

[Terry Wallace]

I’d take 2 of the 40 gig ports on each 6610 and make them a stack
Then I’d take the other 2 ports off the 6610s and make an mlag to your arista. That way you get failover stack connections.

 

[spali]

I’d take 2 of the 40 gig ports on each 6610 and make them a stack
Then I’d take the other 2 ports off the 6610s and make an mlag to your arista. That way you get failover stack connections.

I'm new to mlag, but aren't the links then limited to 8x 10g per switch?

 

[Joao Paulo Mello]

Guys. Finally I got my hands on my ICX -6610-48p and it could not be different being a Cisco guy I'm having some trouble to configure the 2 rear stacking ports.
I want to use as a regular port to connect to my windows 10 box but no joy.
I'm using the fllowing hardware and cables:
NIC: MCX354A-FCBT 649281-B21 656089-001 VPI FDR 40GbE Mellanox from HP 544QSFP MCX354A-FCBT 649281-B21 656089-001 VPI FDR 40GbE Mellanox OEMFirmware | eBay
Cable: Netapp 112-00178 5M 16ft QSFP-QSFP External SAS Cable X6559-R6 from NetApp X 6559 R 6 SAS Cable 5 M QSFP for sale online | eBay

Windows 10 recognized the NIC right away but no link and the ICX6610 says the same:

#show interfaces ethernet 1/2/1
40GigabitEthernet1/2/1 is down, line protocol is down
Port down for 1 hours 13 minutes 24 seconds
Hardware is 40GigabitEthernet, address is cc4e.24f8.ef2c (bia cc4e.24f8.ef5d)
Interface type is 40Gig Fiber
Configured speed 40Gbit, actual unknown, configured duplex fdx, actual unknown
Configured mdi mode AUTO, actual unknown
Member of L2 VLAN ID 1, port is untagged, port state is BLOCKING
BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
Link Error Dampening is Disabled
STP configured to ON, priority is level0, mac-learning is enabled
Flow Control is enabled
Mirror disabled, Monitor disabled
Mac-notification is disabled
Not member of any active trunks
Not member of any configured trunks
No port name

The same happens with the 1/2/6.
The device came factory reset so it was pretty clean, that's my running config, pretty short:

#show running-config
Current configuration:
!
ver 08.0.10mT7f3
!
stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-trunk 1/2/1 to 1/2/2
stack-trunk 1/2/6 to 1/2/7
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
enable telnet authentication
!
username root password .....
!
!
!
!
!
!
!
!
!
interface management 1
ip address 192.168.1.1 255.255.255.0
!
interface ve 1
ip address 10.10.240.250 255.255.255.0
!
!
!
!
!
!
!
!
!
end



Any help would be greatly appreciated.