Solaris 11.3 network slow...

Discussion in 'Solaris, Nexenta, OpenIndiana, and napp-it' started by kemic, Feb 12, 2017.

  1. kemic

    kemic New Member

    Joined:
    Aug 15, 2015
    Messages:
    14
    Likes Received:
    4
    Hey everyone, I've been racking my brain for hours now trying to figure this out. I have a fresh install of Solaris 11.3, setup a simple ZFS pool (4 disks raidz2), configured everything and started rsyncing some data over. That's when I noticed the absolute garbage speed of my 1Gb connection. I first thought it was my pool config but I was able to rule that out with some dd tests. I started monitoring the connection with dlstat and noticed the receive speed never goes over 2MB/s! For the sake of troubleshooting, I forced the port on the switch to do 100Mbps instead of 1Gbps and somehow my speeds went up to 9MB/s. :| I'm pretty stuck here... Anyone have any suggestions before I install another OS to determine if it's a hardware problem? I'm not on site but will check/replace the cable too tomorrow...
     
    #1
  2. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,288
    Likes Received:
    758
    I would first check the pure network with iperf between server and client
     
    #2
  3. kemic

    kemic New Member

    Joined:
    Aug 15, 2015
    Messages:
    14
    Likes Received:
    4
    Thanks for the quick reply!

    Code:
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0- 1.0 sec  4.50 MBytes  37.7 Mbits/sec
    [  3]  1.0- 2.0 sec  4.38 MBytes  36.7 Mbits/sec
    [  3]  2.0- 3.0 sec  6.62 MBytes  55.6 Mbits/sec
    [  3]  3.0- 4.0 sec  2.62 MBytes  22.0 Mbits/sec
    [  3]  4.0- 5.0 sec  8.12 MBytes  68.2 Mbits/sec
    [  3]  5.0- 6.0 sec  4.12 MBytes  34.6 Mbits/sec
    [  3]  6.0- 7.0 sec  3.88 MBytes  32.5 Mbits/sec
    [  3]  7.0- 8.0 sec  2.75 MBytes  23.1 Mbits/sec
    [  3]  8.0- 9.0 sec  3.50 MBytes  29.4 Mbits/sec
    [  3]  9.0-10.0 sec  3.38 MBytes  28.3 Mbits/sec
    [  3] 10.0-11.0 sec  2.38 MBytes  19.9 Mbits/sec
    [  3] 11.0-12.0 sec  4.62 MBytes  38.8 Mbits/sec
    [  3] 12.0-13.0 sec  5.25 MBytes  44.0 Mbits/sec
    [  3] 13.0-14.0 sec   896 KBytes  7.34 Mbits/sec
    [  3] 14.0-15.0 sec  5.00 MBytes  41.9 Mbits/sec
    [  3] 15.0-16.0 sec  3.88 MBytes  32.5 Mbits/sec
    [  3] 16.0-17.0 sec  3.88 MBytes  32.5 Mbits/sec
    [  3] 17.0-18.0 sec  2.75 MBytes  23.1 Mbits/sec
    [  3] 18.0-19.0 sec  3.62 MBytes  30.4 Mbits/sec
    [  3] 19.0-20.0 sec  4.12 MBytes  34.6 Mbits/sec
    [  3] 20.0-21.0 sec  1.17 MBytes  9.79 Mbits/sec
    [  3] 21.0-22.0 sec  2.86 MBytes  24.0 Mbits/sec
    ...
    Here's some test from two physical servers (one CentOS and the other is the problematic Solaris server) in my network, both on the same subnet.

    Code:
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0- 1.0 sec   114 MBytes   957 Mbits/sec
    [  3]  1.0- 2.0 sec   113 MBytes   945 Mbits/sec
    [  3]  2.0- 3.0 sec   112 MBytes   938 Mbits/sec
    [  3]  3.0- 4.0 sec   112 MBytes   937 Mbits/sec
    [  3]  4.0- 5.0 sec   113 MBytes   947 Mbits/sec
    [  3]  5.0- 6.0 sec   112 MBytes   937 Mbits/sec
    [  3]  6.0- 7.0 sec   113 MBytes   945 Mbits/sec
    [  3]  7.0- 8.0 sec   112 MBytes   936 Mbits/sec
    [  3]  8.0- 9.0 sec   113 MBytes   945 Mbits/sec
    [  3]  0.0- 9.1 sec  1.00 GBytes   943 Mbits/sec
    
    Here's a test from the same CentOS physical to a CentOS virtual (also on the same subnet).

    Something for sure is off...

    Code:
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0- 1.0 sec  9.62 MBytes  80.7 Mbits/sec
    [  3]  1.0- 2.0 sec  9.12 MBytes  76.5 Mbits/sec
    [  3]  2.0- 3.0 sec  9.00 MBytes  75.5 Mbits/sec
    [  3]  3.0- 4.0 sec  9.38 MBytes  78.6 Mbits/sec
    [  3]  4.0- 5.0 sec  9.38 MBytes  78.6 Mbits/sec
    [  3]  5.0- 6.0 sec  9.12 MBytes  76.5 Mbits/sec
    [  3]  6.0- 7.0 sec  9.12 MBytes  76.5 Mbits/sec
    [  3]  7.0- 8.0 sec  9.38 MBytes  78.6 Mbits/sec
    [  3]  8.0- 9.0 sec  9.12 MBytes  76.5 Mbits/sec
    [  3]  9.0-10.0 sec  9.25 MBytes  77.6 Mbits/sec
    [  3] 10.0-11.0 sec  9.25 MBytes  77.6 Mbits/sec
    [  3] 11.0-12.0 sec  9.12 MBytes  76.5 Mbits/sec
    [  3] 12.0-13.0 sec  9.25 MBytes  77.6 Mbits/sec
    [  3] 13.0-14.0 sec  9.12 MBytes  76.5 Mbits/sec
    
    Here's another test, same as the first problematic one, with the port forced to 100Mbit on the switch...

    EDIT:

    Here's the command I ran server (Solaris) side:
    Code:
    # iperf -s -i 1
    ------------------------------------------------------------
    Server listening on TCP port 5001
    TCP window size: 1000 KByte (default)
    ....
    
    Here's the command I ran client (CentOS) side:
    Code:
    # iperf -n 1024M -i 1 -c <IP>
    ------------------------------------------------------------
    Client connecting to <IP>, TCP port 5001
    TCP window size: 85.0 KByte (default)
    ------------------------------------------------------------
    ...
    
     
    #3
    Last edited: Feb 12, 2017
  4. kemic

    kemic New Member

    Joined:
    Aug 15, 2015
    Messages:
    14
    Likes Received:
    4
    Going to swap out the network cable before blowing away the OS. Will report back tomorrow.
     
    #4
  5. kemic

    kemic New Member

    Joined:
    Aug 15, 2015
    Messages:
    14
    Likes Received:
    4
    I changed the network cable and i'm getting much more acceptable speeds:

    Code:
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0- 1.0 sec  86.1 MBytes   722 Mbits/sec
    [  3]  1.0- 2.0 sec  84.5 MBytes   709 Mbits/sec
    [  3]  2.0- 3.0 sec  86.0 MBytes   721 Mbits/sec
    [  3]  3.0- 4.0 sec  86.4 MBytes   725 Mbits/sec
    [  3]  4.0- 5.0 sec  85.9 MBytes   720 Mbits/sec
    [  3]  5.0- 6.0 sec  85.4 MBytes   716 Mbits/sec
    [  3]  6.0- 7.0 sec  87.6 MBytes   735 Mbits/sec
    [  3]  7.0- 8.0 sec  86.0 MBytes   721 Mbits/sec
    [  3]  8.0- 9.0 sec  89.9 MBytes   754 Mbits/sec
    [  3]  9.0-10.0 sec   109 MBytes   916 Mbits/sec
    [  3] 10.0-11.0 sec  86.1 MBytes   722 Mbits/sec
    [  3]  0.0-11.6 sec  1.00 GBytes   741 Mbits/sec
    
    See not quite as good as the other gigabit tests in my environment but I'll take it for now...
     
    #5
Similar Threads: Solaris network
Forum Title Date
Solaris, Nexenta, OpenIndiana, and napp-it Solaris network slow (vmxnet3) Nov 21, 2018
Solaris, Nexenta, OpenIndiana, and napp-it Vlan persistence in Solaris 11.3 Jan 8, 2020
Solaris, Nexenta, OpenIndiana, and napp-it solaris 11.4 and samsung 960 evo Oct 29, 2019
Solaris, Nexenta, OpenIndiana, and napp-it OmniOS vs Solaris 11.4 SMB default behaviour Jun 6, 2019
Solaris, Nexenta, OpenIndiana, and napp-it Solaris 11.4 meta devices for DDT Apr 10, 2019

Share This Page