Half decent "All in One" Build

Discussion in 'DIY Server and Workstation Builds' started by pricklypunter, Nov 10, 2015.

  1. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,883
    Likes Received:
    1,509
    You should run the tests w/out SLOG and with sync=always and compare. From what I saw you ran sync=standard with NO slog and then ran sync=always with a SLOG yet some of your tests in sync=standard may not have actually been using the on-disk ZIL.

    Apples to apples would be comparing sync=always without and sync=always with.

    If you're doing the test over your network then your network will play a factor in performance too.
     
    #21
    pricklypunter likes this.
  2. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,288
    Likes Received:
    758
    That is correct.
    Sync Write and an Slog is not a performance option of ZFS it is a security option when you need a transaction safe write behaviour that is far better than the security level that you can achieve with a hardware raid controller and a BBU.
     
    #22
    pricklypunter likes this.
  3. pricklypunter

    pricklypunter Well-Known Member

    Joined:
    Nov 10, 2015
    Messages:
    1,546
    Likes Received:
    441
    Ok, so having re-run the benchmark without a Slog, I can see that it obviously does make a huge difference to sync write performance. It also pretty much confirms what @T_Minus and @gea have mentioned. The Slog device I was using is simply not fast enough to keep up with the spindles.

    The results of the same test from my Laptop over a 1Gbps iSCSI link to the pool VM, but this time without a Slog device fitted and sync=always:

    [​IMG]

    The results of the same test from my Windows VM to the pool VM using 2 x bonded 10Gbps NIC's (VMXNet3) at each end, but this time without a Slog device fitted and sync=always:

    [​IMG]

    All I can say is that I'm astounded at the difference between sync and async write performance without a Slog. I knew there would be a hit in performance with sync writes, but holy sh1t batman...

    Ok, so riddle me this...

    Why are the 4k reads going up with sync=always?
     
    #23
  4. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,883
    Likes Received:
    1,509
    The SSD is < latency than the spindle pool so 4K is > with a SLOG, even if it's not the fastest SLOG due to how ZFS does transactions and flushing to the pool when utilizing a SLOG. (That's my understanding at-least)
     
    #24
  5. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,288
    Likes Received:
    758
    Your first question must be: do I need sync write?

    The answer is no for an SMB filer and the answer is yes
    for VMs with old filesystems or databases where you need safe transactions.

    If you decide that you need sync, you can use the onpool ZIL.
    With enterprise SSDs and powerloss protection like Intel S3610 or better this is perfect.

    If you have spindels the sync write performance can go down to 10% of the nonsync value.
    You can now use a dedicated Slog (Dram based or Flashbesed) to reduce the gap to unsync writes.
     
    #25
  6. pricklypunter

    pricklypunter Well-Known Member

    Joined:
    Nov 10, 2015
    Messages:
    1,546
    Likes Received:
    441
    Cheers guys. Luckily, I don't actually need a SLOG for this build as the storage pool is just using iSCSI to provide block devices for media file storage and some shared space for file backups. There's no fast transaction databases and no iSCSI or NFS VM datastores to worry about etc, if a file gets banjaxed during a copy, it's no biggie. I was merely curious if I could achieve, particularly given the budget nature of the build, similar sync write performance as I was seeing with async by adding a SLOG device to the pool. I misunderstood a vital part of how the ZIL, RAM and SLOG interact to achieve a sync write, but hey it's all a learning experience!
     
    #26
  7. cferra

    cferra New Member

    Joined:
    Jun 27, 2016
    Messages:
    11
    Likes Received:
    0
    Looks like you are having success - I've got the same board - eBay special and for the life of me I can't seem to get it to boot windows after the install from USB drive from anything other then the onboard sata connectors. I've gotten nowhere with booting from a sata 3 hba or the onboard lsi sas controller. Any input would Be great before I pull what little hair I have out left!

    Thanks!
     
    #27
  8. pricklypunter

    pricklypunter Well-Known Member

    Joined:
    Nov 10, 2015
    Messages:
    1,546
    Likes Received:
    441
    I boot ESXi from a USB pen drive. I store my VM's on an 850evo SSD connected to one of the onboard 6Gbps ports and then use the on-board LSI2008 to connect to my backplane and spindles. The onboard LSI2008 is in passthru to my storage VM where I have a ZFS pool configured. Upon which I have several ZVols that I pass to my other VM's via iSCSI. Both the onboard LSI2008 and the external M1015 are flashed to IT mode, without boot bios. I'm not booting anything from either of them :)

    What exactly are you trying to achieve?
     
    #28
  9. cferra

    cferra New Member

    Joined:
    Jun 27, 2016
    Messages:
    11
    Likes Received:
    0
    I have recently been upgrading to 10gb so my thought was to make it a nice storage server - sis oft Sandra showed me that the blue sata ports were only running in 3gb mode even after setting the limited bios to gen 3 mode so I thought ok I'll just boot from the sas controller. Nope sorry can't do it - updated the sas controller firmware to the latest bios and ir mode firmware and even flashed the Lsi efi bios - still no luck - and of course I can't set it to boot legacy mode because the bios doesn't have that option - so then I thought maybe I'll boot from a sata 3 pci controller - nope still no luck. I'm kinda at a loss - I'm about ready to ship this thing back. It's a shame the components are really a good mix but the limited bios makes things difficult. Have you been able to boot fr an hba or the sas controller at all? Plus no ipmi--- not a deal breaker but sucks a bit
     
    #29
  10. cferra

    cferra New Member

    Joined:
    Jun 27, 2016
    Messages:
    11
    Likes Received:
    0
    Can't even use it for a pfsense build - pfsense doesn't support uefi boot and no legacy mode on the bios. Ugh!
     
    #30
  11. pricklypunter

    pricklypunter Well-Known Member

    Joined:
    Nov 10, 2015
    Messages:
    1,546
    Likes Received:
    441
    Can you list what components you are using? Mainboard, RAM, HBA/ Controllers etc?
     
    #31
  12. cferra

    cferra New Member

    Joined:
    Jun 27, 2016
    Messages:
    11
    Likes Received:
    0
    Sure - Right now I'm just keeping it simple toying with it trying to get it configured etc.. - nothing extra is installed - 4 gig Samsung ECC DDR 3 1600 MHz ram - The same OEM Inventec board as you with the onboard LSI 9211-8i SAS controller - ADATA SATA 3 SDD - I actually was finally able to get it to boot with the onboard SAS controller after a LOT of trial and error.

    The reason I was so focused on getting it to boot from the SAS controller is because despite being described as 2x SATA 3 ports on board and even setting the mode to GEN 3 - I was unable to get the full 6GBs - the SSD would only link at 3GBs, (I connected the drive to one of the blue SATA ports onboard) but the SAS controller is 6GBs and I had some spare SAS to SATA breakout cables.

    To get it to boot finally I had to actually upgrade the firmware on the onboard SAS contrller with the latest LSI p20 Firmware and bios and then not only that - I had to flash the EFI bios on to the board. Once I was able to successfully flash the EFI bits on to the card, I had to remove the boot drive designation in the LSI firmware config page and set the onboard storage controller setting in the bios to UEFI Only, I was then able to boot Windows 10.

    What a pain in the butt - this board will ONLY boot efi mode - no way to switch it to legacy mode at all.. ugh.

    No IPMI is another major bummer too.. I'm spoiled by HP iLO.

    Now - to buy some more ram, replace this older SSD.. move my array out of my main server and wait for that 10gb nic.
     
    #32
    Last edited: Jul 9, 2016
Similar Threads: Half decent
Forum Title Date
DIY Server and Workstation Builds Supermicro X8DTE-F borked? See's half ram, and wont boot usb/cd - help Jul 23, 2013
DIY Server and Workstation Builds Configuring "new" BladeCenter S 8886 assistance Dec 19, 2015

Share This Page