All flash storage server

Discussion in 'NAS Systems and Networked Home and SMB Software' started by Paladin, Aug 25, 2017.

  1. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    I am in the process of building a storage server with the following components:

    Intel 2u chassis
    Intel S2600GL motherboard
    2 x E5-2660
    128gb ram
    18 x 500gb dual port NetApp sm1625
    Chelsio T580-CR connected to Cisco 3064
    3 x Intel dual port backplane
    X x Hba's as I would like to avoid using the expander

    The server will be used for general file storage for a render farm and iscsi boot for the render nodes. The ssd's are giving strange performance numbers while testing single drives in s2016 using crystaldiskmark with a 4gb test file:
    single port ReFS 521/552
    single port NTFS 353/197
    dual port ReFS 883/769
    dual port NTFS 896/529

    What is the recomended os for the project? s2016 with storage spaces, freenas, ubuntu with mdadm or else?

    Thanks!
     
    #1
  2. K D

    K D Well-Known Member

    Joined:
    Dec 24, 2016
    Messages:
    1,259
    Likes Received:
    261
    Subscribing


    Sent from my iPhone using Tapatalk
     
    #2
  3. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,279
    Likes Received:
    1,263
    I'm not a windows guy so I wouldn't suggest anything windoes for storage.

    I'd be more comfortable with FreeNAS (ZFS), Napp-IT (OmniOS or?? with ZFS) or Ubuntu with mdadm.
     
    #3
    wildchild likes this.
  4. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    Installed Ubuntu and started playing, just need to find out how to setup multipath io.
    ubuntu 1 drive sp.JPG
     
    #4
  5. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    If we talk about modern storage, especially OpenSource Storage we talk about ZFS. This means mainly Solaris, the source of ZFS and even today the fastest/most feature complete ZFS. There is a free Solaris fork now (Illumos) with most of its features. There is Open-ZFS as a common platform among BSD, Illumos, OSX and Linux, There is the BSD family that took over most of ZFS years ago. Now we have ZFS on Linux that tries to make it mainstream.

    Btrfs on Linux is an effort to add some of the innovations of ZFS like CopyOnWrite or checksums to its former native filesystem ext4. This is the same with ReFS that tries to add some ZFS features to ntfs. Both are caught by its former "ideas" how to manage storage or by "home wants" how to extend storage. With btrfs you lack stability, with ReFS you lack the simplicity or performance of ZFS,

    If you want to try the the genuine "ZFS Experience" intended by Sun, the leading server firm when ZFS was introduced - the only one to fight against Windows NT as a full featured server OS with its own NFS and SMB service among others at that time, first try Solaris or its free forks, then BSD and then the adventure game named "ZFS on Linux"

    This is how I see it!
     
    #5
    Last edited: Aug 25, 2017
  6. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,338
    Likes Received:
    384
    I see oracle will let you download Oracle Solaris 11.3 but does it cost to run, i.e must I buy a license?
     
    #6
  7. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    Yes for commercial use (min 1k USD/year),
    no for demo an development - but without updates

    .... Someone must pay to win the Americas Cup
     
    #7
    pgh5278 and RyC like this.
  8. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,338
    Likes Received:
    384
    Well I like updates... going to go look for prices.
     
    #8
  9. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    #9
  10. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,338
    Likes Received:
    384
    Not really easy. I gotta call sales and ask. Just curious. I like the idea of being at the source of ZFS.
     
    #10
  11. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    Its Irony.
    Last year there was a buy now button where you was able to select cpu cores and just buy online. The reference to the buy now is on the page, the buy button not.

    I wonder quite often why they (they=Dell, HP, IBM, Oracle,..) make buying something often so complicated and user unfriendly.
     
    #11
    gigatexal likes this.
  12. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    Installed Solaris 11.3 and napp-it. Now to setup multipath, seems a lot easier than on ubuntu

    -multipath enabled using # stmsboot -e
     
    #12
    Last edited: Aug 26, 2017
  13. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    #13
    Last edited: Aug 26, 2017
  14. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    I followed the second guide, but after using the
    # mpathadm list LU command, I don't get anything :(

    On Windows, after adding the mpio feature, I had to add the drive in the mpio settings, and after a reboot, they were working in dual port configuration.
     

    Attached Files:

    #14
  15. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,279
    Likes Received:
    1,263
    There's no "set in stone" price, this way they can offer the same software/license/whatever to much smaller businesses while not upsetting their enterprise clients paying 100x more for nearly the same thing. A % of their clients businesses grow, and with it their licensing cost usually do too until they're at enterprise paying level, etc... (This is obviously over simplified but the enterprise service/license/software market is hugely profitable for a reason.)
     
    #15
  16. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    small update, edited the scsi_vhci.conf file and added:

    scsi-vhci-failover-override =
    "NETAPP X452_1625500MCSG", "f_sym",
    "XYZ ABC", "f_sym";

    now I can see the drive but with only one path. :)
     

    Attached Files:

    #16
  17. dragonme

    dragonme Member

    Joined:
    Apr 12, 2016
    Messages:
    234
    Likes Received:
    24
    anyone messed with openattic? I think CERN used it for some peatbyte storage... its based on zfs I think..

    Features
    Storage Management and Monitoring
    The code is based on a modern and extensible architecture, built with proven web technologies like AngularJS, Bootstrap and the Django web application framework.

    • Wizard-based workflows for common tasks
    • Supports multiple file systems for file-based storage management (XFS, ext3/4, Btrfs, ZFS on Linux)
    • LVM2 support
    • Syncronous volume mirroring via DRBD ®
    • CIFS (via Samba)
    • NFS
    • iSCSI and Fibre Channel (via LIO)
    • Volume snapshots (LVM2, ZFS)
    • Automatic monitoring of performance data with Nagios/Icinga and PNP4Nagios
    Ceph Management and Monitoring
    • Customizable Ceph Cluster dashboard (cluster health, cluster performance data)
    • Supports managing multiple Ceph clusters from the same UI
    • Ceph Pool management (view/create/delete) and Monitoring
    • Ceph Pool Erasure code profile management
    • RADOS Block device management (view/create/delete/map) and monitoring
    • Object Storage Daemon (OSD) management (view)
    • CRUSH Map editor
    Other features
    • Installation packages for multiple Linux distributions (Debian/Ubuntu/RHEL/CentOS/SUSE)
    • Multi-user support
    • Multi-node support (one UI to manage multiple openATTIC instances)
    • API Recorder for easy application development/debugging
    Missing a feature?
    openATTIC is under active development. If you're missing a particular feature, please let us know!

    © 2017 openATTIC
     
    #17
  18. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    Multipath finaly working.

    It turns out that Solaris didn't see the RMS25CB080 raid module, it seems that the drivers are missing. I also found out that it has some bandwith limitations, so will try and skip that one. Curently using 2 x hp h222 flashed to lsi p20, will need to downgrade to p19 as I read on napp-it FAQ that p20 has performance issues with fast ssd's.

    Now to do some tests with 4 drives :)
     
    #18
    gigatexal likes this.
  19. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,778
    Likes Received:
    597
    Fine that you got it working

    P20 was reported buggy between 20.0.0.0 and 20.0.0.4.
    Current 20.0.0.7 seems ok
     
    #19
  20. Paladin

    Paladin New Member

    Joined:
    Sep 26, 2016
    Messages:
    20
    Likes Received:
    4
    Great, I will stay on p20 in that case, as it is 20.0.0.7. Here are the first test with 4 drives in raidz. Will need to wait for the switch to arrive in order to test iscsi and smb.
     

    Attached Files:

    #20
    K D and gigatexal like this.
Similar Threads: flash storage
Forum Title Date
NAS Systems and Networked Home and SMB Software iXsystems FreeNAS Certified Flash storage array Feb 24, 2017
NAS Systems and Networked Home and SMB Software Synology All-Flash box.. review please? Dec 1, 2016
NAS Systems and Networked Home and SMB Software NAS and Storage Options Apr 12, 2018
NAS Systems and Networked Home and SMB Software Oracle ZFS Storage Appliance available - What to test? Apr 3, 2018
NAS Systems and Networked Home and SMB Software NVMe Storage Solution Apr 1, 2018

Share This Page