Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Craig Curtin

Member
Jun 18, 2017
94
19
8
58
what wrong with my running config, please? Cant connect to my web management, or if connect to my switch my router management but can ping to external site from the router now. Running in router mode



!
aaa authentication snmp-server default local
aaa authentication web-server default local
aaa authentication login default local
ip default-network 10.0.0.0/23
ip dns server-address 10.0.0.1 10.0.0.5
ip route 0.0.0.0/0 10.0.0.1
!
username *** password .....
!
!
clock timezone us Alaska
!
!
ntp
server 134.0.16.1 minpoll 5
server 162.159.200.1
server 217.114.59.66
server 185.83.169.27
!
!
web-management enable ethe 1/1/1
!
interface ethernet 1/1/1
ip address 10.0.0.4 255.255.255.0
!
!
end
What code are you running on the switch ?

What VLANs have you defined ? - none in the config you have posted.

What is this switch ?

Craig
 

ramicio

Member
Nov 30, 2022
32
4
8
You need to post up your config and also provide a little more detail - are you saying you are connecting the Intel card to the breakout cable - what is then mean to be providing DHCP to the Intel card and which switch port is that connected to ?

Craig
Hello. I don't have very much understanding of this stuff, so if you would be able to tell me exactly what I need to do to be able to give you this information, that would be appreciated.

No breakout cable. A QSFP+ DAC. Only 1 port on the switch shows any sign of electrical life.

Means of providing DHCP? Probably my router. Connected to one of the regular RJ45 ports. Doesn't matter. If I use a static IP it's nothing but trouble.
 

gregsachs

Active Member
Aug 14, 2018
503
168
43
Interpreting TDR results?
(Disclaimer: I"m just missing somethin stupid)
Ok, I have a cable running to a closet, USW-Flex in there to forward PoE to an AP and such.
One cable run downstream of the Flex will only link at 100M, not 1GB. If I jumper around the USW and have the 6450 patched to that run, a TDR test tells me to look at local pair B.
I've looked here:
and here:
and for the life of me I can't decide if that means orange or green, given all my keyholes are wired 568B.
Just trying to figure out where to focus.
 

Cobra0101

New Member
Nov 22, 2022
10
0
1
What code are you running on the switch ?

What VLANs have you defined ? - none in the config you have posted.

What is this switch ?

Craig
ICX 6645 - running in router mode
no vlans defind yet as only got as far as set default route when i tested to see if i could access the web admin page and failed


ICX6450-24 Router#sho ve
version System status
ICX6450-24 Router#sho version
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
UNIT 1: compiled on Nov 27 2017 at 13:41:09 labeled as ICX64R08030q
(9855884 bytes) from Primary ICX64R08030q.bin
SW: Version 08.0.30qT313
Boot-Monitor Image size = 786944, Version:10.1.05T310 (kxz10105)
HW: Stackable ICX6450-24
==========================================================================
UNIT 1: SL 1: ICX6450-24 24-port Management Module
Serial #: 2ax5o2jk68e
License: ICX6450_PREM_ROUTER_SOFT_PACKAGE (LID: H4CKTH3PLN8)
P-ENGINE 0: type DEF0, rev 01
==========================================================================
UNIT 1: SL 2: ICX6450-SFP-Plus 4port 40G Module
==========================================================================
800 MHz ARM processor ARMv5TE, 400 MHz bus
65536 KB flash memory
512 MB DRAM
STACKID 1 system uptime is 3 minute(s) 29 second(s)
The system : started=cold start
 

Cobra0101

New Member
Nov 22, 2022
10
0
1
also when i try and create a virtual interface

ICX6450-24 Router(config-vlan-1)#Int ve 1
Error - invalid virtual ethernet interface number.
ICX6450-24 Router(config)#Error - invalid virtual ethernet interface number.
Invalid input -> Error - invalid virtual ethernet interface number.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,616
2,810
113
32
fohdeesha.com
also when i try and create a virtual interface

ICX6450-24 Router(config-vlan-1)#Int ve 1
Error - invalid virtual ethernet interface number.
ICX6450-24 Router(config)#Error - invalid virtual ethernet interface number.
Invalid input -> Error - invalid virtual ethernet interface number.
please follow the guide more closely, that's the wrong command
 

ramicio

Member
Nov 30, 2022
32
4
8
Is anyone able to help me get to the bottom of the issues I'm having with [probably] the switch? I need to know if I should be asking for a refund for the switch, or other parts, or if there's really nothing wrong anywhere and I'm just an idiot.
 

itronin

Well-Known Member
Nov 24, 2018
1,037
669
113
Denver, Colorado
Is anyone able to help me get to the bottom of the issues I'm having with [probably] the switch? I need to know if I should be asking for a refund for the switch, or other parts, or if there's really nothing wrong anywhere and I'm just an idiot.
1) Did you follow the setup config licensing guide v2 in the first post of this thread?
2) Did you follow the steps to disable stacking - you'll need to do that in order to use the rear 40Gbe and 40Gbe breakout ports (they are different - are you aware of this?)
3) Have you learned how to pull the config from your switch? (you'll need to be able to ssh into the switch or use the console) I believe you've previously been asked to provide the config. The config is super duper helpful (really critical) for remote T/S.

If you haven't done item 1 above then at a minimum you're switch probably isn't where it needs to be for steps 2 and 3. if you purchased the switch from ebay and expected it to work out of the box its VERY UNLIKELY that it will.

One other bit of information, the intel (and oem intel) 10/40Gbe nics can be very picky about DACS and how they self identify. not saying that's an issue here but it is something to be aware of.

there's a world of difference between being ignorant and being an idiot and the first step is figuring out what you know and don't know before you decide to describe yourself as one or the other.
 

ramicio

Member
Nov 30, 2022
32
4
8
1. Yes
2. Yes
3. No. I was asked to do this, and I do not know how. I asked how to do this, and got no reply. I can SSH into the switch. What do I need to do to supply that information?

Regarding Intel being picky about DACs. I did the xl710_unlocker thing (which is supposed to be for optics, right?). Regardless, I think people keep glossing over the fact that only 1 of the 40 gig ports show any sign of life, and the one that does is riddled with nothing but errors. I don't mean to be rude about that last part, but please tell me what I specifically need to do on the switch to provide the required information.
 

itronin

Well-Known Member
Nov 24, 2018
1,037
669
113
Denver, Colorado
1. Yes
2. Yes
3. No. I was asked to do this, and I do not know how. I asked how to do this, and got no reply. I can SSH into the switch. What do I need to do to supply that information?

Regarding Intel being picky about DACs. I did the xl710_unlocker thing (which is supposed to be for optics, right?). Regardless, I think people keep glossing over the fact that only 1 of the 40 gig ports show any sign of life, and the one that does is riddled with nothing but errors. I don't mean to be rude about that last part, but please tell me what I specifically need to do on the switch to provide the required information.
k.

In step 2 near the end is the following:
Code:
show run
There should no longer be any stack-trunk commands, or any commands referencing stack ports. That's pretty much it! You can now use the ports on the rear as normal. However, see the next section on how they are laid out.

you will need to cut and paste the output and post it here. my advice is to wrap it in "code" and in a "spoiler" so you don't post a huge reply and force people to scroll through it.

Yes I saw your comments and yeah you may have a bad switch - however I don't want to be rude and ASSUME something so I recommend starting at the beginning and provide the config. You have missed going through items 1 and 2 how to pull the config so I provided the linkage on how to do that.
 

ramicio

Member
Nov 30, 2022
32
4
8
Excellent, thank you. Here it is:

Code:
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-24p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
stack disable
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
aaa authentication web-server default local
aaa authentication login default local
hostname switcheroo
ip dhcp-client disable
!
no telnet server
username root password .....
!
!
!
!
!
!
!
!
!
interface ethernet 1/1/3
inline power
!
interface ethernet 1/1/6
inline power
!
interface ethernet 1/1/7
inline power
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
!
interface ethernet 1/3/3
speed-duplex 10G-full
!
interface ethernet 1/3/4
speed-duplex 10G-full
!
interface ethernet 1/3/5
speed-duplex 10G-full
!
interface ethernet 1/3/6
speed-duplex 10G-full
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
speed-duplex 10G-full
!
interface ve 1
ip address 192.168.1.5 255.255.255.0
!
!
!
!
!
!
!
!
!
end
 

itronin

Well-Known Member
Nov 24, 2018
1,037
669
113
Denver, Colorado
Excellent, thank you. Here it is:
with the QSFP DAC plugged into each of the left side connections please provide the switch's show interface output. It may also be helpful to show status output (from your OS) as well for each port but that is up to you.

recommend using the same cable for each interface test.
 

ramicio

Member
Nov 30, 2022
32
4
8
with the QSFP DAC plugged into each of the left side connections please provide the switch's show interface output. It may also be helpful to show status output (from your OS) as well for each port but that is up to you.

recommend using the same cable for each interface test.
Code:
SSH@switcheroo>show interface ethernet 1/2/1
40GigabitEthernet1/2/1 is up, line protocol is up
  Port up for 1 hour(s) 17 minute(s) 51 second(s)
  Hardware is 40GigabitEthernet, address is 748e.f8fe.93f6 (bia 748e.f8fe.940f)
  Interface type is 40Gig Fiber
  Configured speed 40Gbit, actual 40Gbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual none
  Member of L2 VLAN ID 1, port is untagged, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is enabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  MTU 1500 bytes, encapsulation ethernet
  300 second input rate: 224 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 1976 bits/sec, 2 packets/sec, 0.00% utilization
  7685 packets input, 826324 bytes, 0 no buffer
  Received 272 broadcasts, 7404 multicasts, 9 unicasts
  33 input errors, 0 CRC, 0 frame, 0 ignored
  0 runts, 30 giants
  502846 packets output, 56757049 bytes, 0 underruns
  Transmitted 181923 broadcasts, 319738 multicasts, 1185 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled

Egress queues:
Queue counters    Queued packets    Dropped Packets
    0              502861                   0
    1                   0                   0
    2                   0                   0
    3                   0                   0
    4                   0                   0
    5                   2                   0
    6                   0                   0
    7                   0                   0
Code:
tim@fileserver:~$ ethtool enp98s0
Settings for enp98s0:
        Supported ports: [ FIBRE ]
        Supported link modes:   40000baseCR4/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  40000baseCR4/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 40000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
Cannot get wake-on-lan settings: Operation not permitted
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

Code:
SSH@switcheroo>show interface ethernet 1/2/6
40GigabitEthernet1/2/6 is up, line protocol is up
  Port up for 32 second(s)
  Hardware is 40GigabitEthernet, address is 748e.f8fe.93f6 (bia 748e.f8fe.9414)
  Interface type is 40Gig Fiber
  Configured speed 40Gbit, actual 40Gbit, configured duplex fdx, actual fdx
  Configured mdi mode AUTO, actual none
  Member of L2 VLAN ID 1, port is untagged, port state is FORWARDING
  BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
  Link Error Dampening is Disabled
  STP configured to ON, priority is level0, mac-learning is enabled
  Openflow is Disabled, Openflow Hybrid mode is Disabled,  Flow Control is enabled
  Mirror disabled, Monitor disabled
  Mac-notification is disabled
  Not member of any active trunks
  Not member of any configured trunks
  No port name
  MTU 1500 bytes, encapsulation ethernet
  300 second input rate: 216 bits/sec, 0 packets/sec, 0.00% utilization
  300 second output rate: 1416 bits/sec, 1 packets/sec, 0.00% utilization
  10 packets input, 964 bytes, 0 no buffer
  Received 0 broadcasts, 10 multicasts, 0 unicasts
  0 input errors, 0 CRC, 0 frame, 0 ignored
  0 runts, 0 giants
  64 packets output, 6290 bytes, 0 underruns
  Transmitted 20 broadcasts, 44 multicasts, 0 unicasts
  0 output errors, 0 collisions
  Relay Agent Information option: Disabled

Egress queues:
Queue counters    Queued packets    Dropped Packets
    0                  68                   0
    1                   0                   0
    2                   0                   0
    3                   0                   0
    4                   0                   0
    5                   0                   0
    6                   0                   0
    7                   0                   0
Code:
tim@fileserver:~$ ethtool enp98s0
Settings for enp98s0:
        Supported ports: [ FIBRE ]
        Supported link modes:   40000baseCR4/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  40000baseCR4/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 40000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
Cannot get wake-on-lan settings: Operation not permitted
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

I don't really know what's going on and I don't know how to articulate it.

What was in the machine before was a 10G SFP+ NIC. It was enp98s0. So I put this thing in (XL710) and it takes the same name. I've since put in a different 10G NIC and it took the name of enp129s0. All is fine and works there. Got its MAC address in my router for a static lease (192.168.1.2 is the IP of the machine I desire). When I mess around with this XL710 on enp98s0 and I go to kill it with ifdown, it says "DHCPRELEASE of 192.168.1.2 on enp98s0 to 192.168.1.1 port 67 (xid=0x7d5d72e3)" This is after assigning it a static IP in /etc/network/interfaces. I really don't get what's going on, and I am about to just wipe the OS install (which I've had maintained for like 14 years now, and would be a HUGE pain in the butt to start over).
 

itronin

Well-Known Member
Nov 24, 2018
1,037
669
113
Denver, Colorado
I don't really know what's going on and I don't know how to articulate it.
...


STOP! Take a deep breath.

I reviewed your previous posts. I'd like to get some clarity first. I'm going to give you some suggestions in a bit so bear with me.

Your interface output looks "fine" even the small number of giants (hoping they're just 1522 and not something else - which could be coming form something else on your network). I'm going with the test on the 1/2/6 was simply brief as the uptime on it is short. - please confirm

based on the show int I'd say both 40gbe ports are fine.

for each interface test did you have good link / status "blinky lights" on your NIC?
You previously reported you had no electrical activity on the bottom port - is that still the case?

What was in the machine before was a 10G SFP+ NIC. It was enp98s0. So I put this thing in (XL710) and it takes the same name. I've since put in a different 10G NIC and it took the name of enp129s0. All is fine and works there. Got its MAC address in my router for a static lease (192.168.1.2 is the IP of the machine I desire). When I mess around with this XL710 on enp98s0 and I go to kill it with ifdown, it says "DHCPRELEASE of 192.168.1.2 on enp98s0 to 192.168.1.1 port 67 (xid=0x7d5d72e3)" This is after assigning it a static IP in /etc/network/interfaces. I really don't get what's going
on, and I am about to just wipe the OS install (which I've had maintained for like 14 years now, and would be a HUGE pain in the butt to start over).
STOP again! Please don't wipe your 14 year maintained system. If it has been maintained that long I'd either consider it "prod" or of significant sentimental value. Put it back on your network the way it was and leave it alone until you've finished testing the switch.

You are experimenting and troubleshooting - use something else for this process.

Beg, borrow, steal another system. Throw an OS on it (clean install) and let's troubleshoot the rest of the ports on your switch. yeah?

seriously. Don't EFF around with the system you've been using. Use something else to finish testing out the switch.

Next, I saw a comment that made me think you want to rely on DHCP for testing. Bad idea right now.
80% of most network issues are physical layer.
If you rely on DHCP you may question whether DHCP is behaving or you have something else going on. Remove DHCP from the equation. Use static IP's for the remainder of the testing and on a CLEAN system.
that way you can focus on do I have a bad port, do i have a bad cable.
simplify your troubleshooting by simplifying the number of variables in any given test.
 

kh78

New Member
Mar 31, 2020
29
6
3
Hi all,

Hoping you might be able to offer some guidance on moving from L2 to L3 code (I'm doing this really just as a learning exercise, actually do my routing in a VM).

I have a batch of VLANs, with a separate management VLAN (VLAN 10). That VLAN had the management IP on it for inband access, and I also have a separate physical switch for the mgmt network, cabled to the OOB management port on the back.

I've just moved to running the routing l3 image code and figured I'd learn about setting up Multi-VRF and also management-vrf.

I setup a vrf (MGMT), and a router-interface ve 10 on VLAN 10. The router-interface ve 10, is forwarding into vrf MGMT. All is well thus far.

I then applied management-vrf MGMT which was accepted fine, and when I check with 'show management-vrf' it does indeed appear to have worked.

MY QUESTION
The OOB Management port, does not report itself as belonging to my newly defined management vrf (MGMT). I thought that was part of what the 'management-vrf MGMT' command was meant to do, i.e. forward the OOB physical management ethernet interface in to whatever VRF you had defined as the
management-[B]vrf?

Basically everything below points to my inband ve 10 interface being in the managment vrf correctly, but not the OOB interface. How do I get the OOB to forward into the defined managment-vrf?

show who[/B]
Code:
SSH connections (inbound):
1      established, client ip address 192.168.1.50, server hostkey RSA, user is USER, privilege super-user, in config mode
        using vrf default-vrf.
        you are connecting to this session
        10 second(s) in idle
^^^ That particular SSH session is to the IP address that is configured on the OOB management port, and as you can see, it reports it's in the default-vrf.

sh ip int
Code:
sh ip int
Interface           IP-Address      OK?  Method    Status             Protocol   VRF
Eth mgmt1           192.168.1.171 YES  manual    up                 up         default-vrf
Ve 10               192.168.1.173 YES  manual    up                 up         MGMT
show vrf MGMT
Code:
VRF MGMT, default RD 4:10, Table ID 1
Configured as management-vrf
IP Router-Id: 10.10.10.10
  Interfaces:
    ve10

  Address Family IPv4
    Max Routes: 200
    Number of Unicast Routes: 1