Napp-It AIO Performance Troubleshooting

Discussion in 'Solaris, Nexenta, OpenIndiana, and napp-it' started by coolrunnings82, Mar 5, 2014.

  1. coolrunnings82

    coolrunnings82 Active Member

    Joined:
    Mar 26, 2012
    Messages:
    399
    Likes Received:
    88
    Environment:
    HP Proliant ML110 G7
    Xeon E3-1220
    16gb DDR3 ECC RAM
    4x 500gb Western Digital RE4 (2x ZFS mirrored sets)
    1x 500gb Western Digital RE4
    IBM M1015 flashed to IT mode
    ESXi 5.1 Update 2
    OpenIndiana 151-A8 - Allocated 8gb RAM
    Windows 7 Pro x64 VM's x 2 - Allocated 4gb RAM each

    Problem:
    1. Super slow disk performance on a relatively intermittent basis.
    2. Gnome interface showing up with no menu bar at the top nor anything at the bottom - just a desktop with a couple folders.

    What I've Tried/Checked:
    • Checked SMART data on drives. All check out with nothing out of the ordinary and no reallocated or pending bad sectors
    • Ran a scrub on the pool. Performance starts out at 2mb/s and caps out around 55mb/s. Normal?
    • Ensured that NFS Sync was disabled for the VM pool.

    What I need:
    • Suggestions for troubleshooting the slowdown.
    • Recommendations of tools/procedure that can individually test the drives to see if one is actually experiencing issues that are slowing down the overall performance.
    • Suggestions for fixing Gnome interface.

    Thanks for any help!!!
     
    #1
  2. mrkrad

    mrkrad Well-Known Member

    Joined:
    Oct 13, 2012
    Messages:
    1,241
    Likes Received:
    51
    hard drives without a BBWC/FBWC is going to be painfully slow!

    Also ensure your host is set to max power/disable C1e/disable C-states, and your guest vm's likewise are set to maximum power mode (no power savings!) it makes a huge difference even to this day.

    And disable IPV6 on esxi 5.1, and disable flow control.

    I'm not sure what nic you are using but that might be a problem - ssh into the esxi server and check /var/log/vmkernel.log and /var/log/vmkwarning.log or just do a dmesg and check for errors.

    Latency warnings are a sign of death with esxi! You should not see them unless you are doing a SCSI hot-add/snapshot!

    There are some tweaks you can do to increase queue depth - but my suggestion is to baseline benchmark using 1 VM on the host - then add more vm's and watch performance drop like a bandit as esxi's fair-queueing comes into play!
     
    #2
  3. coolrunnings82

    coolrunnings82 Active Member

    Joined:
    Mar 26, 2012
    Messages:
    399
    Likes Received:
    88
    Thanks for the tips. I'll check tonight. Also I've not received any latency warnings from ESXi, just dealt with extreme latency causing me issues...
     
    #3
  4. coolrunnings82

    coolrunnings82 Active Member

    Joined:
    Mar 26, 2012
    Messages:
    399
    Likes Received:
    88
    In the vmkwarning.log file I see lots of these:
    VFAT: 4346: Failed to flush file times: Stale file handle.
    LinScsi: SCSILinuxQueueCommand:1193:queuecommand failed
    Pretty sure that's when I rebooted the Openindiana VM.

    Not much other stuff that looks bad that I could identify... Waiting till nobody else is on then I'll change the NIC settings.
     
    #4
Similar Threads: Napp-It Performance
Forum Title Date
Solaris, Nexenta, OpenIndiana, and napp-it napp-it on OmniOS on ESXi - performance? Jul 19, 2017
Solaris, Nexenta, OpenIndiana, and napp-it Napp-it SuperStorage Server 6048R-E1CR36L Performance May 30, 2016
Solaris, Nexenta, OpenIndiana, and napp-it omnios/napp-it: Improving write performance May 13, 2015
Solaris, Nexenta, OpenIndiana, and napp-it Is there plans to integrate a VM manager into Napp-IT? Yesterday at 11:09 AM
Solaris, Nexenta, OpenIndiana, and napp-it napp-it 14b AIO upgrade path? Jan 12, 2020

Share This Page