Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,345
    Likes Received:
    1,080
    it goes in the serial console port hopefully you didn't burn anything out lmao
     
    #2221
  2. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
  3. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,345
    Likes Received:
    1,080
    as they say in the business, wrong hole
     
    #2223
  4. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
    Yeah, I totally misread your instructions:
    Connect to the switches serial/console port on the front using a program like Putty (9600 8N1), and connect any of the normal switch ports to your network (do NOT use the dedicated management port

    Shame.
     
    #2224
    fohdeesha likes this.
  5. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,345
    Likes Received:
    1,080
  6. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
    Yah, i'm definitely in the right hole now.. still no console output though
     
    #2226
  7. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,345
    Likes Received:
    1,080
    you might have fried the adapter, hard to say
     
    #2227
  8. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
    Just as well I have another one coming as I wasn't sure if this one was going to turn up
     
    #2228
  9. infoMatt

    infoMatt Member

    Joined:
    Apr 16, 2019
    Messages:
    39
    Likes Received:
    18
    Why on earth should a link light come on if a serial converter is connected to a eth port? Hmmm... that doesn't sound right to me at all (but I don't want to try with my adapter, sorry :p)...
     
    #2229
  10. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    56
    Likes Received:
    24
    Because a link light will come on with a loopback, which a serial adapter will certainly be able to provide. Problem is serial isn't really pulling/using power to the same level Ethernet is, so yeah, that could've fused the thing.

    It works both ways too; I've personally seen a console port on a 6610-48 that very likely had an ethernet run plugged into it because the port was dead. Switch worked fine and the guy I sold it to was able to get a serial connection rigged up (I didn't have the knowledge for it) and confirm it was working and able to use it.
     
    #2230
  11. infoMatt

    infoMatt Member

    Joined:
    Apr 16, 2019
    Messages:
    39
    Likes Received:
    18
    Err... nope... or at least, I don't think so... because the ethernet link will come up if the board detects a signal on the cable, but looking at the pinout for the Serial-RJ45 the TX and RX are on pin 3 and 6, or where the green/green-stripe pair would be in T568B... and they are isolated from each other (the TX switch side is terminated on the base of a mosfet, either in the serial bridge chip or in a buffer, in both case it is seen as a capacitance to ground, to me it seems strange that a ethernet PHY could detect it as a remote peer...).
    Pins 4 and 5 are shorted to ground (blue pair), pins 1 and 2 are RTS and DTR, both an output from the switch, so they couldn't in any way generate the ethernet carrier signal... strange as hell...

    Very well possible; as stated, both signal pairs of a 100Base-T will hit both output pins of the serial port of the switch, or one input pin ad one output pin, but on the ethernet signalling the voltage is applied only between the wires of every single pair; there is no voltage between each pair and "ground" (in fact, unless they were using shielded wires, there's no common groud between two devices connected by an ethernet cable!).
     
    #2231
  12. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    56
    Likes Received:
    24
    All of which bets are off if you have a fried serial controller creating the loop. I've actually seen the link light thing stuck on with a management port that was accidentally plugged into on a 3750X, so I assume it's something getting borked along the way.
     
    #2232
    fohdeesha likes this.
  13. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
    The serial cable still gets detected in windows if that's any consolation

    I'll have to wait for the new cable to arrive because I get no output on either switches management port
     
    #2233
  14. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
    New cable arrived, and it works

    Either I broke the other one or it never worked
     
    #2234
  15. raab

    raab New Member

    Joined:
    Aug 24, 2019
    Messages:
    12
    Likes Received:
    3
    And one of them has a faulty PoE board, great :)

    Code:
    PoE: Stack unit 1 PS 1, Internal Power supply  with 370000 mwatts capacity is up
    PoE Info: Adding new 54V capacity of 370000 mW, total capacity is 370000, total free capacity is 370000
    PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/24 detected. Initializing....
    PoE Event Trace Log Buffer for 2000 log entries allocated
    PoE Event Trace Logging enabled...
    PoE Error: Device 0 failed to start on PoE module.
    PoE Error: Device 1 failed to start on PoE module.
    Resetting module in slot 1 again to recover from dev fault
    PoE Info: Hard Resetting in slot 1....
    PoE Info: Resetting module in slot 1....completed.
    PoE Error: Device 0 failed to start on PoE module.
    PoE Error: Device 1 failed to start on PoE module.
    
     
    #2235
  16. Churchill

    Churchill Admiral

    Joined:
    Jan 6, 2016
    Messages:
    766
    Likes Received:
    170
    Is there anyway I can find out why my speeds are not 40GB or so? For some unholy reason my rsync copies are hovering around 45MB/s which is abysmal when I should be getting a helluva lot more. Disks are Western Digital Reds, nothing else is going on between the arrays the disks are reading/writing from. CPU is low, memory is low, networking is barely a blip in speed. I cannot push the 40GB speed that I was hoping for.
     
    #2236
  17. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    56
    Likes Received:
    24
    So, what's the array? What speed are you getting writing a large file to the destination? Speed reading from a large file from the source? Have you tried running iperf between the two endpoints and seen what sort of speeds you're getting? Do you have any other connections that might be getting used (like an old gigabit link that you're keeping for redundancy)?

    If you're not seeing errors on the output of "show interface ethe 1/2/X" and the log isn't showing anything either, it's unlikely to be a basic switching issue, but we'd need to know a lot more about source and destination before we could hazard a guess on anything else.
     
    #2237
  18. Churchill

    Churchill Admiral

    Joined:
    Jan 6, 2016
    Messages:
    766
    Likes Received:
    170
    Array is a pair of Supermicro 2U servers with dual 6 core/12 thread Xeon 2600, RAM is 128GB in each. Both are controlled by SAS controllers and the disks are SATA.

    I ran iperf between the two and only when I was pushing 10+ threads was I getting anywhere near 30GB/s speed between the two.

    Both running 40GB hp cards reflashed to mellanox stock in ethernet mode. Jumbo frames is turned on for both.

    I'm shocked I'm not getting at least 100MB/s from these transfers as the disks can easily sustain 150-200MB/s in transfer speeds.
     
    #2238
  19. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    56
    Likes Received:
    24
    Alright, so even if there was something wrong with the disks, there's still a problem with the network throughput.

    Are both devices in the same VLAN? If not, where's the routing being done, on the switch or elsewhere? If you've got spare cards/10G ports, can you pull the 40G links and test over the 10G and see if you get better throughput, the same, or worse? (Either iperf or file transfer, preferably both.)

    Are we sure the NICs are running a full PCIe x8 electrical slot, and is it PCIe 2.0 or 3.0? (I think it has to be 3.0, but I have a vague recollection of motherboard PCIe generation being controlled by the CPU generation as well - just don't know if that was these CPUs or the E3s.) Why I ask is that PCIe 2.0 x8 tops out at 4GB/sec, which is 32 gigabit - suspiciously close your max transfer speed with multiple iperf threads.
     
    #2239
  20. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,345
    Likes Received:
    1,080
    30gbps iperf is good. you're not going to do much better than that without newer CPUs and some kernel tweaking. 45MB/s file shares sounds like NFS with sync=enabled or something
     
    #2240
Similar Threads: Brocade Series
Forum Title Date
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019
Networking Brocade VDX 6720 - what do I need to know? Sep 16, 2019
Networking anyone have Brocade CER 2024C latest firmware Sep 14, 2019
Networking Guest VLAN on Brocade 6450 Jul 18, 2019

Share This Page