Storage Server For Media and Video Editing Setup

Discussion in 'FreeBSD and FreeNAS' started by GUOGELIN, Sep 6, 2019.

?

What OS to use?

  1. Windows Server

  2. FreeNAS

  3. FreeBSD

  4. Ubuntu

  5. CentOS

  6. Others (Please specify in the comments)

Results are only viewable after voting.
  1. GUOGELIN

    GUOGELIN New Member

    Joined:
    Sep 6, 2019
    Messages:
    5
    Likes Received:
    0
    Hello everyone,
    I am new to the forum, and I don't know if this is the right place to post this here.
    I am looking to build a storage server for my studio, which I will be using for a few applications.
    I would like to know what kind of OS and setup should I use. I don't know much Linux commands, but if there is a guide, I can follow it and do some command lines. I tried Windows Server 2019, but the GUI version is not working, and even using Powershell is giving me errors after errors. Freenas is easy to setup, but I have no idea what drive to use as cache and what to use as log.
    The most important ones are video editing, media server, personal backup server, and other random stuff.
    Currently I have an Unraid server that we use for these applications, but the speed of Unraid is just too slow for what we do.
    So for video editing right now we have the needs for two editors to edit straight from the server, and want to have the possibility to expand that to 4-5 editors in the near future. Right now with Unraid we have to pull the footage from the server to our workstations first, and then send them back after we are done with editing. We are shooting 4K footage, and need real time editing in Adobe Premier with out any lag.
    As for media I am leaning towards Plex. I have an Unraid server that have Plex right now, but when ever plex is doing something, like parity check or even just the mover running from the SSD cache to spinning drives, the movies are unwatchable, always lagging.
    This server needs to be really fast, and very reliable with redundancies.
    The hardware I have now:
    Supermicro X10DRi
    2X Intel E5-2620V4
    4X 32GB of DDR4 ECC memory
    15X 2TB WD Blacklable Drives
    4X 240GB SSD
    2X Intel 4610 NVME 1.6TB U.2 with PCIE converter
    Intel X550 10G network card
    LSI 9305-24i
    24 Bay Supermicro Chassis with individual sata port backplane
    I got all of these servers for free from a friends company because they bought too much of these, so I don't mind to change or upgrade some of these parts if needed. But I think these are really good hardware for a storage server.
    And I have a raid card lying around but have no use to it, I can add that in if needed.
    Also I got Samsung 970s in both workstations, and 10G network to all workstations and servers.
    Please ask any question if you need to know more about any hardware or my needs if you didn't understand.
    Thank you.
     
    #1
  2. Mishka

    Mishka Member

    Joined:
    Apr 30, 2017
    Messages:
    73
    Likes Received:
    10
    Whats the network usage like whilst editing? just making sure that its not a network limitation on the desktop computers.
     
    #2
  3. GUOGELIN

    GUOGELIN New Member

    Joined:
    Sep 6, 2019
    Messages:
    5
    Likes Received:
    0
    All computers are running on 10G network now, and there are only two computers/workstations connected to the network. For Plex I am using either my Xbox One X or Ps4 Pro, both are Gigabit network (I think), so shouldn't be a problem with network.
    When I transfer files to my Unraid server, the speed is about 600-700MB/s, which is the limit of the SSDs I used in the Unraid server, they are two Samsung 860 sata drives in Raid 1.
     
    #3
  4. pricklypunter

    pricklypunter Well-Known Member

    Joined:
    Nov 10, 2015
    Messages:
    1,506
    Likes Received:
    430
    Just a thought, and I am probably barking up the wrong tree, but you mentioned that you are pulling your 4k streams to the xbox one x? If you have a low core count or a particularly low speed CPU, you might be working it really hard, or even pegging it, transcoding the audio. Have a look at the server stats while your video is playing :)
     
    #4
  5. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,184
    Likes Received:
    708
    There are several options

    Disk performance
    If single disk performance is too slow (depends on your use case ex hd vs 4k) you need a realtime raid where sequential performance scale with number of datadisks and random performance with number of raids (raid60 0r 2 x ZFS vdev hast twice the iops of raid6 or single ZFS vdev). Your current setup always use a single disk only.

    If you want realtime Raid, the best raid regarding data security and scalability is ZFS. Despite its superiour security, ZFS is fast especially due its superiour rambased read and write caches.

    If you want ZFS, your next question is OS and SMB server. For the OS you can use Solaris or a free Solaris fork where ZFS comes from (best integration) and where you can use the into ZFS/OS embedded multithreaded SMB server, mostly the fastest one with Windows ntfs alike ACL.

    The other option is Free-BSD (a Unix like Solaris) with SAMBA as SMB server or Linux with SAMBA. If you do nor want to manage at CLI, you need a webmanaged appliance.

    The options on Solarish
    -Commercial Oracle Solaris + my napp-it
    -OpenSource OmniOS (a free Solaris fork for production use) + my napp-it
    -Nexentastor (commercial ZFS appliance)

    The options on Free-BSD
    On Free-BSD you cannot just use a normal Free-BSD with its regular updates and fixes but a distribution with web-ui like FreeNAS or TrueNAS as its commercial option.

    The options on Linux
    ex OMV but not on same level regarding ZFS like the Unix options.

    From Setup
    I would create two ZFS pools from mirrors from same disks (2TB pool and very fast NVMe pool). The samller SSDs are useless for a server. Your RAM is large enough so you do not need an SSD read cache. For video editing you do not need a protection of the write cache (slog).

    Performance wise, you can expect around 500MB/s from a pool of 7 x mirror of the old 2TB disks and nearly 10G from the NVMe under best conditions.
     
    #5
  6. GUOGELIN

    GUOGELIN New Member

    Joined:
    Sep 6, 2019
    Messages:
    5
    Likes Received:
    0
    Hi,
    My Unraid server has two E5-2620 V4 as well, 8 core at 2.1Ghz each. So this shouldn't be a problem. Plex in Xbox is only a client, all transcode is done in the server.
     
    #6
  7. GUOGELIN

    GUOGELIN New Member

    Joined:
    Sep 6, 2019
    Messages:
    5
    Likes Received:
    0
    Hi,
    So for software I am more towards FreeNAS as it has a nice GUI, I don't know much about Linux command lines so it is very important for me. Or if there is a very detailed step by step guide, maybe I can follow that as well.
    For the small SSDs I am planning just to use two of them as boot drive in raid 1, and leave the other two alone.
    For the NVME drives you said to create a separate pool with them, will they be in the same volume as the spinning hard drives? Or they will be a different volume?
     
    #7
  8. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,184
    Likes Received:
    708
    FreeNAS based on Free-BSD is ok.

    My alternative is napp-it, a Web-UI for a default Oracle Solaris, OmniOS or OpenIndiana operating system, https://napp-it.org/doc/downloads/napp-it.pdf

    Managing ZFS from commandline is a pain, even when only two commands are involved, see illumos: manual page: zfs.1m and illumos: manual page: zpool.1m. Additionally you must manage an SMB server either the ZFS internal one on Solarish or SAMBA on Free-BSD or Linux + an Initiator. Webmanagement is much easier.

    On ZFS you have pools, where you create filesystems or zvols (a filesystem treated like a raw disk). The term volume can mean a pool or a zvol so I would avoid.

    You build a slower pool from the disk and a faster one from NVMe and share each. On a client you can either mount the slower or the faster share.
     
    #8
  9. GUOGELIN

    GUOGELIN New Member

    Joined:
    Sep 6, 2019
    Messages:
    5
    Likes Received:
    0
    I will definitely check out napp-it, it sounds like a really good solution. FreeNAS is nice, but there are limitations.
     
    #9
Similar Threads: Storage Server
Forum Title Date
FreeBSD and FreeNAS Storage Server advice Dec 18, 2015
FreeBSD and FreeNAS Need guidance on what OS/storage solution to choose for my media server Nov 5, 2015
FreeBSD and FreeNAS Help new build with Supermicro Storage Jan 1, 2019
FreeBSD and FreeNAS Linux Docker Host with FreeNAS network storage Jan 31, 2017
FreeBSD and FreeNAS Detaching ZFS pools in RAIDZ1 - functional external storage? Jan 9, 2017

Share This Page