wow really? I may need to update the main post, I swore the last time I tested 7250's, they would boot fine without even any fan plugged in at all
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Yeah, not sure why. I'll try to connect with the console cable and see if I can get a hint as to what's going on. Surprised it kept rebooting.
Update: Looks like the Sunon and other slower fans aren't fast enough for the switch. Received this through the console:
FATAL ERROR: At least one good FRU fan should be present in the system
System is shutting down!!!
Last edited:
Sounds like they added a sanity check for minimum tach, probably due to aging. (Some of these fans as they age, will spin slower due to increased resistance before failing.) Or because of us. Either or. Could also be because of changing ODM parts.
Obviously could always throw a multiplier on the tach line, but definitely not a great idea, because that would prevent ramping under load. ("I expect a max 15k RPM, fan says it's at 16k, so don't give it more because of overspeed!") If we can figure out the minimum, I can probably find some alternative parts.
Well I somehow managed to find not one but two DAC cables that will not work on either an ICX 7150 or an ICX 6450, I thought I would leave the info here so others avoid making the same mistake. The switches do detect the modules, they just do not link.
#1 Belkin F2CX036-05M Cable, SFP+, 10GBASE Direct Attached Passive TWNAX, 5M
This DAC is marketed as a cisco compatible part, I could not get it working with any of the ruckus/brocade switches I have. I was able to get it working for directly connecting two Intel X520 adapters. Below is the dump of the module using ethtool.
#2 Amphenol 571540001 0.5m SFP+ 10GbE Direct Attach Passive Copper Cable twinaxial
This DAC was the same as above, it did not work in the switches, but did work in a NIC.
Lastly is a cable that I have been using for some time and it has been working in practically everything I've got. They were included with my connectx-2 cards a while back so I don't know much about them.
If anyone has any recommendations I am looking to pick up a few extra cables 0.5M, 1M, and 5M, I do use optical but I would like the DAC for troubleshooting and flexibility purposes. For instance if I order on FS should I be getting brocade compatible cables or Cisco being that I do have that one cisco-tyco cable that works fine? I was thinking of going with these Cisco SFP-H10GB-CU3M 3 Meter 10G SFP+ DAC Twinax Cable 37-0961 MNPA19-XTR | eBay
Alternatively if there is a way to recode the modules, I'm all ears. I saw how Fohdeesha did this with the monoprice qfsp cables but that was on an ICX6610 which I do not have access to. I know this can be also be done with specialty hardware such as a Revelprog EEPROM programmer with sfp adapter as well but I didn't think it was worth the cost for what will possibly be a one time use.
#1 Belkin F2CX036-05M Cable, SFP+, 10GBASE Direct Attached Passive TWNAX, 5M
This DAC is marketed as a cisco compatible part, I could not get it working with any of the ruckus/brocade switches I have. I was able to get it working for directly connecting two Intel X520 adapters. Below is the dump of the module using ethtool.
Code:
root@pve1:~# ethtool -m eno2
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x21 (Copper pigtail)
Transceiver codes : 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
Transceiver type : Passive Cable
Encoding : 0x00 (unspecified)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 0km
Length (SMF) : 0m
Length (50um) : 0m
Length (62.5um) : 0m
Length (Copper) : 5m
Length (OM3) : 0m
Passive Cu cmplnce. : 0x03 (unknown) [SFF-8472 rev10.4 only]
Vendor name : OEM
Vendor OUI : 00:40:20
Vendor PN : SFP-H10GB-CU5M
Vendor rev :
Option values : 0x00 0x00
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : CSC10G53053
Date code : 12071900
root@pve1:~# ethtool -m eno2 hex on
Offset Values
------ ------
0x0000: 03 04 21 00 00 00 00 00 04 00 00 00 67 00 00 00
0x0010: 00 00 05 00 4f 45 4d 20 20 20 20 20 20 20 20 20
0x0020: 20 20 20 20 00 00 40 20 53 46 50 2d 48 31 30 47
0x0030: 42 2d 43 55 35 4d 20 20 20 20 20 20 03 52 00 1d
0x0040: 00 00 00 00 43 53 43 31 30 47 35 33 30 35 33 20
0x0050: 20 20 20 20 31 32 30 37 31 39 30 30 00 00 00 b5
0x0060: 80 00 11 e6 16 55 db 1d 6c f7 fd 8a d2 d0 29 73
0x0070: c4 7f 5c 00 00 00 00 00 00 00 00 00 2a 7c 35 a7
0x0080: 43 4f 50 51 41 41 34 4a 41 41 33 37 2d 30 39 36
0x0090: 30 2d 30 32 56 30 32 20 01 00 46 00 00 00 00 c9
0x00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00c0: 53 46 50 2d 48 31 30 47 42 2d 43 55 35 4d 20 20
0x00d0: 20 20 20 20 30 38 00 00 00 00 00 00 00 00 00 b3
0x00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00#2 Amphenol 571540001 0.5m SFP+ 10GbE Direct Attach Passive Copper Cable twinaxial
This DAC was the same as above, it did not work in the switches, but did work in a NIC.
Code:
root@pve1:~# ethtool -m eno2
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x21 (Copper pigtail)
Transceiver codes : 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
Transceiver type : Passive Cable
Encoding : 0x00 (unspecified)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 0km
Length (SMF) : 0m
Length (50um) : 0m
Length (62.5um) : 0m
Length (Copper) : 1m
Length (OM3) : 0m
Passive Cu cmplnce. : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]
Vendor name : Amphenol
Vendor OUI : 78:a7:14
Vendor PN : 571540001
Vendor rev : W
Option values : 0x00 0x00
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : APE16310011MJV
Date code : 160808
root@pve1:~# ethtool -m eno2 hex on
Offset Values
------ ------
0x0000: 03 04 21 00 00 00 00 00 04 00 00 00 67 00 00 00
0x0010: 00 00 01 00 41 6d 70 68 65 6e 6f 6c 20 20 20 20
0x0020: 20 20 20 20 00 78 a7 14 35 37 31 35 34 30 30 30
0x0030: 31 20 20 20 20 20 20 20 57 20 20 20 01 00 00 5a
0x0040: 00 00 00 00 41 50 45 31 36 33 31 30 30 31 31 4d
0x0050: 4a 56 20 20 31 36 30 38 30 38 20 20 00 00 00 07
0x0060: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x0070: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x0080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x0090: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x00a0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x00b0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x00c0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x00d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x00e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x00f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ffLastly is a cable that I have been using for some time and it has been working in practically everything I've got. They were included with my connectx-2 cards a while back so I don't know much about them.
Code:
root@pve1:~# ethtool -m eno1
Identifier : 0x03 (SFP)
Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)
Connector : 0x21 (Copper pigtail)
Transceiver codes : 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
Transceiver type : Passive Cable
Encoding : 0x00 (unspecified)
BR, Nominal : 10300MBd
Rate identifier : 0x00 (unspecified)
Length (SMF,km) : 0km
Length (SMF) : 0m
Length (50um) : 0m
Length (62.5um) : 0m
Length (Copper) : 2m
Length (OM3) : 0m
Passive Cu cmplnce. : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]
Vendor name : CISCO-TYCO
Vendor OUI : 00:40:20
Vendor PN : 1-2053783-6
Vendor rev : R
Option values : 0x00 0x00
BR margin, max : 0%
BR margin, min : 0%
Vendor SN : TED1635HR5H
Date code : 16082900
root@pve1:~# ethtool -m eno1 hex on
Offset Values
------ ------
0x0000: 03 04 21 00 00 00 00 00 04 00 00 00 67 00 00 00
0x0010: 00 00 02 00 43 49 53 43 4f 2d 54 59 43 4f 20 20
0x0020: 20 20 20 20 00 00 40 20 31 2d 32 30 35 33 37 38
0x0030: 33 2d 36 20 20 20 20 20 52 20 20 20 01 00 00 12
0x0040: 00 00 00 00 54 45 44 31 36 33 35 48 52 35 48 20
0x0050: 20 20 20 20 31 36 30 38 32 39 30 30 00 00 00 fd
0x0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 e9 77 70 80
0x0080: 43 4f 50 51 41 42 54 4a 41 41 33 37 2d 31 34 33
0x0090: 30 2d 30 31 56 30 31 20 01 00 46 00 00 00 00 e1
0x00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00c0: 53 46 50 2d 48 31 30 47 42 2d 43 55 32 4d 20 20
0x00d0: 20 20 20 20 31 30 00 00 00 00 00 00 00 00 00 ad
0x00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00If anyone has any recommendations I am looking to pick up a few extra cables 0.5M, 1M, and 5M, I do use optical but I would like the DAC for troubleshooting and flexibility purposes. For instance if I order on FS should I be getting brocade compatible cables or Cisco being that I do have that one cisco-tyco cable that works fine? I was thinking of going with these Cisco SFP-H10GB-CU3M 3 Meter 10G SFP+ DAC Twinax Cable 37-0961 MNPA19-XTR | eBay
Alternatively if there is a way to recode the modules, I'm all ears. I saw how Fohdeesha did this with the monoprice qfsp cables but that was on an ICX6610 which I do not have access to. I know this can be also be done with specialty hardware such as a Revelprog EEPROM programmer with sfp adapter as well but I didn't think it was worth the cost for what will possibly be a one time use.
Can someone please help. I have just updated my 6450p with the latest firmware for the system as well as POE. It is operating well so far with the limited testing I have done. I cant figure out why POE is not working on the switch yet? I have followed fohdeesha's instructions fully.
This is what the config looks like right now:
!
Startup-config data location is flash memory
!
Startup configuration:
!
ver 08.0.30tT313
!
stack unit 1
module 1 icx6450-48p-poe-port-management-module
module 2 icx6450-sfp-plus-4port-40g-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
hostname alitech
ip dhcp-client disable
ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1
!
!
!
clock summer-time
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
!
!
!
!
!
interface ethernet 1/1/1
inline power
!
interface ve 1
ip address 192.168.1.55 255.255.255.0
!
!
!
!
!
!
!
ip ssh permit-empty-passwd yes
!
!
end
To try to solve the problem, I did a google search and landed on here where they have some instructions to turn on POE:
Enabling and disabling Power over Ethernet
Its still not working despite showing up in config that inline power is on the interface. Is there something else I need to do? Perhaps I havent turned it on for all ports?
Please let me know where I am going on.
Also, just to confirm, the ports are all POE auto sensing? (figuring out if the device requires POE or not)
thanks
This is what the config looks like right now:
!
Startup-config data location is flash memory
!
Startup configuration:
!
ver 08.0.30tT313
!
stack unit 1
module 1 icx6450-48p-poe-port-management-module
module 2 icx6450-sfp-plus-4port-40g-module
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
hostname alitech
ip dhcp-client disable
ip dns server-address 192.168.1.1
ip route 0.0.0.0/0 192.168.1.1
!
!
!
clock summer-time
!
!
ntp
disable serve
server 216.239.35.0
server 216.239.35.4
!
!
!
!
!
interface ethernet 1/1/1
inline power
!
interface ve 1
ip address 192.168.1.55 255.255.255.0
!
!
!
!
!
!
!
ip ssh permit-empty-passwd yes
!
!
end
To try to solve the problem, I did a google search and landed on here where they have some instructions to turn on POE:
Enabling and disabling Power over Ethernet
Its still not working despite showing up in config that inline power is on the interface. Is there something else I need to do? Perhaps I havent turned it on for all ports?
Please let me know where I am going on.
Also, just to confirm, the ports are all POE auto sensing? (figuring out if the device requires POE or not)
thanks
What do you get from the show inline power command?
Code:
telnet@ICX7150-24P Switch>show inline power
Power Capacity: Total is 150000 mWatts. Current Free is 120000 mWatts.
Power Allocations: Requests Honored 6 times
Port Admin Oper ---Power(mWatts)--- PD Type PD Class Pri Fault/
State State Consumed Allocated Error
--------------------------------------------------------------------------
1/1/1 On On 5000 10000 802.3at Class 4 3 n/a
1/1/2 Off Off 0 0 n/a n/a 3 n/a
1/1/3 On On 4500 10000 802.3at Class 4 3 n/a
1/1/4 Off Off 0 0 n/a n/a 3 n/a
1/1/5 On Off 0 0 n/a n/a 3 n/a
1/1/6 Off Off 0 0 n/a n/a 3 n/a
1/1/7 On Off 0 0 n/a n/a 3 n/a
1/1/8 Off Off 0 0 n/a n/a 3 n/a
1/1/9 On Off 0 0 n/a n/a 3 n/a
1/1/10 Off Off 0 0 n/a n/a 3 n/a
1/1/11 On On 4200 10000 802.3af Class 3 3 n/a
1/1/12 Off Off 0 0 n/a n/a 3 n/a
1/1/13 Off Off 0 0 n/a n/a 3 n/a
1/1/14 Off Off 0 0 n/a n/a 3 n/a
1/1/15 Off Off 0 0 n/a n/a 3 n/a
1/1/16 Off Off 0 0 n/a n/a 3 n/a
1/1/17 Off Off 0 0 n/a n/a 3 n/a
1/1/18 Off Off 0 0 n/a n/a 3 n/a
1/1/19 Off Off 0 0 n/a n/a 3 n/a
1/1/20 Off Off 0 0 n/a n/a 3 n/a
1/1/21 Off Off 0 0 n/a n/a 3 n/a
1/1/22 Off Off 0 0 n/a n/a 3 n/a
1/1/23 Off Off 0 0 n/a n/a 3 n/a
1/1/24 Off Off 0 0 n/a n/a 3 n/a
--------------------------------------------------------------------------
Total 13700 30000I enabled it for first 8 ports and this is what I see
Code:
telnet@alitech>show inline power
Power Capacity: Total is 740000 mWatts. Current Free is 740000 mWatts.
Power Allocations: Requests Honored 8 times
Port Admin Oper ---Power(mWatts)--- PD Type PD Class Pri Fault/
State State Consumed Allocated Error
--------------------------------------------------------------------------
1/1/1 On Off 0 0 n/a n/a 3 n/a
1/1/2 On Off 0 0 n/a n/a 3 n/a
1/1/3 On Off 0 0 n/a n/a 3 n/a
1/1/4 On Off 0 0 n/a n/a 3 n/a
1/1/5 On Off 0 0 n/a n/a 3 n/a
1/1/6 On Off 0 0 n/a n/a 3 n/a
1/1/7 On Off 0 0 n/a n/a 3 n/a
1/1/8 On Off 0 0 n/a n/a 3 n/a
1/1/9 Off Off 0 0 n/a n/a 3 n/a
1/1/10 Off Off 0 0 n/a n/a 3 n/a
1/1/11 Off Off 0 0 n/a n/a 3 n/a
1/1/12 Off Off 0 0 n/a n/a 3 n/a
1/1/13 Off Off 0 0 n/a n/a 3 n/a
1/1/14 Off Off 0 0 n/a n/a 3 n/a
1/1/15 Off Off 0 0 n/a n/a 3 n/a
1/1/16 Off Off 0 0 n/a n/a 3 n/a
1/1/17 Off Off 0 0 n/a n/a 3 n/a
1/1/18 Off Off 0 0 n/a n/a 3 n/a
1/1/19 Off Off 0 0 n/a n/a 3 n/a
1/1/20 Off Off 0 0 n/a n/a 3 n/a
1/1/21 Off Off 0 0 n/a n/a 3 n/a
1/1/22 Off Off 0 0 n/a n/a 3 n/a
1/1/23 Off Off 0 0 n/a n/a 3 n/a
1/1/24 Off Off 0 0 n/a n/a 3 n/a
1/1/25 Off Off 0 0 n/a n/a 3 n/a
1/1/26 Off Off 0 0 n/a n/a 3 n/a
1/1/27 Off Off 0 0 n/a n/a 3 n/a
1/1/28 Off Off 0 0 n/a n/a 3 n/a
1/1/29 Off Off 0 0 n/a n/a 3 n/a
1/1/30 Off Off 0 0 n/a n/a 3 n/a
1/1/31 Off Off 0 0 n/a n/a 3 n/a
1/1/32 Off Off 0 0 n/a n/a 3 n/a
1/1/33 Off Off 0 0 n/a n/a 3 n/a
1/1/34 Off Off 0 0 n/a n/a 3 n/a
1/1/35 Off Off 0 0 n/a n/a 3 n/a
1/1/36 Off Off 0 0 n/a n/a 3 n/a
1/1/37 Off Off 0 0 n/a n/a 3 n/a
1/1/38 Off Off 0 0 n/a n/a 3 n/a
1/1/39 Off Off 0 0 n/a n/a 3 n/a
1/1/40 Off Off 0 0 n/a n/a 3 n/a
1/1/41 Off Off 0 0 n/a n/a 3 n/a
1/1/42 Off Off 0 0 n/a n/a 3 n/a
1/1/43 Off Off 0 0 n/a n/a 3 n/a
1/1/44 Off Off 0 0 n/a n/a 3 n/a
1/1/45 Off Off 0 0 n/a n/a 3 n/a
1/1/46 Off Off 0 0 n/a n/a 3 n/a
1/1/47 Off Off 0 0 n/a n/a 3 n/a
1/1/48 Off Off 0 0 n/a n/a 3 n/a
--------------------------------------------------------------------------
Total 0 0I too am on the hunt for a reasonably priced set of ears. From what I've gathered there isn't really a readily available compatible ears...
For reference the rack mount kit P/N is "ICX6450-RMK".
Absolutely not necessary, you can enable PoE on just some ports and not on other, that aspect is fine.
Your configuration and the "show inline power" looks good to me, and all that it's necessary really is to enable inline power on the desidered interface(s).
What devices are you trying to power using the 6450? Keep in mind that those switches are 802.3af (PoE) and .3at (PoE+) compliant, so they will source +48V along the pairs of the ethernet cable only if they sense a PD (Powered Deviced) connected; if you plug a "passive" +24V """PoE""" like an external radio or some cheap cameras, they won't pass the PD detection, and thus no power will be delivered.
Rack ears for the 6450 is just a piece of bent sheet metal really, there's not much to it...
The center of the holes is about 11mm from the internal edge (the one touching the frame of the switch), the rail facing lip is 20mm long total (from switch frame edge outwards), and they are 2mm thick.
On mine theres a number written: "B425.00139.015 Rev.00 FOXCONN"
Have you verified the ports you tried these in on your switches are licensed and configured for 10gbE full duplex, eg they link without an issue with other DACs or other optics? Or are these the first things you've tried in these ports? I ask because I've literally used fibrechannel optics in these things, they simply don't care. I'd be surprised if it was indeed the DACs
your config is correct, like infomatt said it looks like the switch isn't seeing any PoE compliant devices on the other end. Note: ubiquiti 24v and other "24v passive PoE" crap is not standards compliant 48v PoE, and will not work with PoE switches without an adapter
Ok I'm just an idiot and have been doing this so long I'm forgetting things, my original 7250 post/notes indeed point out the hardware watchdog and the fact at least one fan needs to be plugged. It would be good if we could pinpoint the rough rpm fans have to meet in speed 1 to not get tagged as failed https://forums.servethehome.com/ind...gbe-40gbe-switching.21107/page-47#post-208624
Thanks fohdeesha....this is the one that confused me.
https://forums.servethehome.com/ind...be-40gbe-switching.21107/page-119#post-241628
The ICX6450 will tag a slow fan as failed but will still boot and run. The ICX7250, at least in my case, will tag it as failed and shut down (actually, it will restart over and over and over again....).