Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
657
244
43
Screw locations on my 24P upstairs:

Top cover front edge: 3
Per Side, front by rack ears:1 -- midway back: 1
Back, bottom edge in each corner: 1 -- middle of back by AUX power input: 1

edit: Oops, the cluster of 3 screws IS the rack ears. There's just 1 in the middle of that holding the cover on.
 

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
657
244
43
Will the ICX6610 PoE model run on the lower spec 250W PSUs? I don't need anything even close to 1kW, I'm drawing something like 15W from my 6450 right now. If possible I'm thinking of picking up a couple of RPS15-E (250W) so I'll have two racked and the fans will spin down. A 250W supply would put me at a more efficient PSU load compared to the 1kW as well. Thoughts?
 

SuperServer

New Member
Feb 1, 2019
15
1
3
So my icx is pulling this error on start up not sure if anyone had this issue before or how to fix it. I hit b but cant stop the boot loader with B it just reenters back into it.

Code:
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
.BOOT ERR: load monitor failed (ERR0)
$

Boot(Mem-Test)>mt
1: [20000000] {Incremental}
Failed at 0: expected 0, actual 400
 
Last edited:

Dreece

Active Member
Jan 22, 2019
503
160
43
I'm no Brocade expert, but that doesn't look promising... :eek:

Do you have home insurance? Sometimes you can claim power-surge damage.

Though not sure if they would class an enterprise switch as a 'home personal-use' product, whatever you do, don't say you work from home, because that's when they'll throw the 'business use' argument at you and you get nada.
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,078
113
33
fohdeesha.com
Will the ICX6610 PoE model run on the lower spec 250W PSUs? I don't need anything even close to 1kW, I'm drawing something like 15W from my 6450 right now. If possible I'm thinking of picking up a couple of RPS15-E (250W) so I'll have two racked and the fans will spin down. A 250W supply would put me at a more efficient PSU load compared to the 1kW as well. Thoughts?

it'll run with the non-PoE supplies, it'll just disable poe (obviously) but you're not going to save any power draw or gain efficiency. the PoE supplies are two seperate DC supplies, the 12v supply that powers the actual switch is the exact same 250w capacity as the non-PoE models. There's a second DC supply around ~54v with a capacity of 750w that is strictly for the PoE daughterboard, it's not going to cause any extra power draw with no PoE devices in use. and even if you do enable a few PoE devices, you're looking at an extra draw of maybe 2 watts per PoE device, so even if the 54v DC rail is doing 50% efficiency because the draw is so far away from it's rated load, you're still barely looking at 2 extra watts of losses
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,078
113
33
fohdeesha.com
So my icx is pulling this error on start up not sure if anyone had this issue before or how to fix it. I hit b but cant stop the boot loader with B it just reenters back into it.

Code:
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
.BOOT ERR: load monitor failed (ERR0)
$

Boot(Mem-Test)>mt
1: [20000000] {Incremental}
Failed at 0: expected 0, actual 400
it's dead. I'd say replace the RAM in any other case but given it seemingly took an indirect strike of some kind (through mains) I seriously doubt that'll fix it
 

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
657
244
43
it'll run with the non-PoE supplies, it'll just disable poe (obviously) but you're not going to save any power draw or gain efficiency. the PoE supplies are two seperate DC supplies, the 12v supply that powers the actual switch is the exact same 250w capacity as the non-PoE models. There's a second DC supply around ~54v with a capacity of 750w that is strictly for the PoE daughterboard, it's not going to cause any extra power draw with no PoE devices in use. and even if you do enable a few PoE devices, you're looking at an extra draw of maybe 2 watts per PoE device, so even if the 54v DC rail is doing 50% efficiency because the draw is so far away from it's rated load, you're still barely looking at 2 extra watts of losses
Sounds like it'll be good enough for me out of the box. Thanks! Looking forward to playing with the SDN features when it arrives :)
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,078
113
33
fohdeesha.com
Hey @fohdeesha any idea why the 7xxx series has the option to run a hypervisor? Curious how they are using it.
it runs the main fastiron app in a VM for isolation/security purposes, you can switch it off and have it run "native" with a bootloader arg for debugging purposes, but shouldn't be ran in production like that. If you got a switch that says "HV disabled" during boot, you need to turn it back on. I've gotten a couple ICX7750s like that, pretty sure they came from the old brocade offices directly (they had the same QR sticker as the others I've gotten from there)
 
  • Like
Reactions: tjk

Churchill

Admiral
Jan 6, 2016
838
213
43
Can i take a Brocade ICX6650 and utilize this as a 4 port 40GBe switch? In this way I truly only need 3 devices plugged into these ports for backups and job replications. I can then take the front part of these switches, drop in a bunch of 10GBE modules and call it day and have 10/40GBE?

I want to simplify my networking stack from 3 switches to 1 if possible with the least amount of equipment due to spacing. Right now it's a 1GB switch, 10GB switch, and 40GB switch. I have a feeling that the answer to my question is YES but it cannot hurt to double check

Answer: YES


Ok looks like I can do this, posted replies.

Any solid QFP modules I can use for this switch?
 

Churchill

Admiral
Jan 6, 2016
838
213
43
Are the runs short? If so you can use NetApp QSFP SAS DACs, they're stupid cheap.

They won't do 56Gb if you're using Infiniband or Mellanox proprietary Ethernet, but they'll happily do 40Gb on the 66xx series.
Less than 3ft for 2 of the runs and then 15' for one run. Nothing over that though.

Those modules still applicable? Not too worried about 56GBe, 40GB is just fine for me so reflashing the cards to generic Mellanox versions is just fine.


also Any 10GBT modules that folks recommend? I have 10GBT everywhere so far and I want to have that one switch that's compatible with everything.
 

itronin

Well-Known Member
Nov 24, 2018
1,234
794
113
Denver, Colorado
Less than 3ft for 2 of the runs and then 15' for one run. Nothing over that though.

Those modules still applicable? Not too worried about 56GBe, 40GB is just fine for me so reflashing the cards to generic Mellanox versions is just fine.
Yes. I only purchased the the short cables (stacking) and the 15' (5m) cables (servers). They work great. specifically X6559-R6, thick cables so not very flexible. About $8.00 - how much ever you want to pay.
 

Churchill

Admiral
Jan 6, 2016
838
213
43
Yes. I only purchased the the short cables (stacking) and the 15' (5m) cables (servers). They work great. specifically X6559-R6, thick cables so not very flexible. About $8.00 - how much ever you want to pay.

I'll need them to curve SLIGHTLY as I do pretty good in my cable management. I think they will do just fine for what I need.

Time to go shopping fo Brocade ICX6650's
 
Last edited:
  • Like
Reactions: itronin

Dreece

Active Member
Jan 22, 2019
503
160
43
Talking of direct-attach cables... to be extra sure, can I use the Brocade 58-0000033-01 QSFP cable to connect the 6610 to a Mellanox X3 card for 40g?

Supposed to work, just wanting verification from someone using this cable right now with a Mellanox X3 :)
 
Last edited:

KFJ

New Member
Mar 11, 2019
10
7
3
OOF! I bought a used ICX-6610 from eBay but it turns out the PoE board was shot (had a hole in an IC and a blown transistor on the PoE board). I was able to update the firmware and image and PoE firmware but the error persisted and of course the error blocked the output of commands like 'show chassis'.

The switch was just saying:
Code:
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
The seller graciously sent another used PoE board which I installed about 40 minutes ago. Now the switch powers on and I can get a console, but the console doesn't load past "Enter 'b' to stop at boot monitor" regardless if I press b per usual or not. The switch never fully boots. Tell me I don't have a $400 brick. Just got my fiber kit today :)o_O

Any tips or tricks to reflash or reset the switch? I still have the old PoE board - at least that let me boot the switch, but I'd kinda like to have the PoE working, but it is optional (I'll have to get another PoE switch).
 

Churchill

Admiral
Jan 6, 2016
838
213
43
Ok after minutes of deliberation I went ahead and bought the following to complete my 40GBE network:

QTY 1: Brocade ICX6650 - $250
QTY 3: HP 659281-B21 10/40GB Adapter - $30/card
QT3: NatAPP SAS QSFP/QSFP Cables - $10/card

So for under $400 I will soon have a 40GB network ready to roll

Now I need some quality 1/10GBE gbics and I can get rid of my other switch.

PS. I love and hate you all....
 
  • Like
Reactions: wsuff and Dreece

tjk

Active Member
Mar 3, 2013
481
199
43
Ok after minutes of deliberation I went ahead and bought the following to complete my 40GBE network:

QTY 1: Brocade ICX6650 - $250
QTY 3: HP 659281-B21 10/40GB Adapter - $30/card
QT3: NatAPP SAS QSFP/QSFP Cables - $10/card

So for under $400 I will soon have a 40GB network ready to roll

Now I need some quality 1/10GBE gbics and I can get rid of my other switch.

PS. I love and hate you all....
What seller did you get your 6650 from? I bought 5 off ebay, 2 of the 5 came damaged and had broken ports.

3 of them were packed nice, 2 of them were packed pretty crappy, makes me scratch my head.