An interesting challenge that requires some help

Discussion in 'Linux Admins, Storage and Virtualization' started by vl1969, Mar 19, 2019.

  1. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    555
    Likes Received:
    62
    Hi everyone.
    I have a chalenge that I would appreciate a little guidance and hand holding for.
    My problem
    I am running a proxmox home server.
    Config in my signature.
    Short outline is a 12 core 128gb ram
    OS disk is zfs mirror on 2 120gb ssd
    3 zfs pools of variose size for data storage.
    A couple of terabytes collection of movies and pics.
    I am just starting setting up my VMs so nothing is running yet except an lxc container with emby.
    Well here is the issue. My emby server having a bit of a difficulty serving my movies as most of then are in mkv format and huge in size. I had to resize the main drive for VM 4 time now as it gets filled with temp files from emby transcodings.
    So my question is.
    If I do not have a dedicated graphics card in the server, besides I am not sure if I can passthrou to lxc. Can I build a headless vm/lxc where I can run a trancoder like handbreak for example and use that to convert my library to mp4?
    What would be the most optimal config for this kind of setup.
    What would be the best program to use . handbreak is great but I have only used it in gui mode.
    An automation option would be nice to.
    Thanks.
     
    #1
  2. BoredSysadmin

    BoredSysadmin Active Member

    Joined:
    Mar 2, 2019
    Messages:
    130
    Likes Received:
    26
    Hmm, I assume the issue is that your client devices don't support mkv and your emby is struggling with transcoding it?
    If so, I solved that particular issue by using no BS approach. My ZFS is on Ivy Bridge Freenas, used to run Plex as jail, but it doesn't support hardware acceleration and besides Ivy Bridge era quicksync is very limited. So I used Appollo lake NUC with J3455 cpu and it hums along nicely with ubuntu 18.04 and latest plex. Enabled hardware transcoding and I was able to serve 3x 1080p streams transcoded at the same time at 65% CPU usage. I am VCP and 20 years experience in IT and I feel no shame trying to have it all in one box. No need to reencode the media files. I just play them as they are and transcoding then necessary isn't an issue.

    Disclaimer: Hardware transcoding with Plex requires Plex Pass.
     
    #2
  3. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    555
    Likes Received:
    62
    Well it dose not really help me.
    I have 2 client.
    An amazone fire stick and Intel NUC with linux mint.
    I do not use plex. I use emby
     
    #3
  4. BoredSysadmin

    BoredSysadmin Active Member

    Joined:
    Mar 2, 2019
    Messages:
    130
    Likes Received:
    26
    #4
  5. Unfadingpyro

    Unfadingpyro New Member

    Joined:
    Sep 17, 2016
    Messages:
    16
    Likes Received:
    12
    You should be able to setup a linux vm and use handbrake to convert the files to MP4. Have you tried converting a file to MP4 to verify its the mkv files causing the issue?

    I used this guide to setup a pc to rip my dvd collection. It looks for disc's in the player, rips them using MakeMKV, then passes it along to Handbrake and dumps the file into my media collection. Its been awhile since i've looked at the scripts for it, but you should be able to adapt it to look through your existing media collection and add them to the Handbrake queue.

    Automatic Ripping Machine | Headless | Blu-Ray/DVD/CD | b3n.org
     
    #5
  6. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    555
    Likes Received:
    62
    Unfadingpyro,
    Thanks. I actually found this script and plan to adapt it if I can.
    Here is the gist of my issues and plan of attack.

    Issue 1. My emby lxc crashes when playing videos. Problem is with emby filling up the OS disk with transcoding temp files.
    Since right now I only have one client I just enlarged the OS disk to 100gb. I have No movies larger than 40gb.
    .. Future plan maybe add an ssd or dedicated hdd to the mix and bind mount it to vm and use that for emby temp files.

    It is not an mkv that cause problem but the size of some of them.

    Issue 2. Since I have a bunch of files like that. I want to setup an automated way for me to transcode easily the one I want to make a uniform format all over and shrink the size along the way.

    I want/need to run this on my server. So. Proxmox lxc preferably headless with WebUI if possible. I will load a webmin there so ui is not really an issue.
    My server has no reasonably good GPA
    Can it handle the transcoding as is?
     
    #6
  7. Continuum

    Continuum Member

    Joined:
    Jun 5, 2015
    Messages:
    77
    Likes Received:
    24
    How does your video storage hook into your container? Bind mount? If so, have you tried creating a temp directory within the bind mount structure and changing transcoding temporary path to the new directory? This is how I have set up my emby container. This setup allowed the use of a 5GB root file system for the container.
     
    #7
  8. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    555
    Likes Received:
    62
    I was thinking about it but I just expand the OS drive of container. It's the same storage.
    Transcoding in emby is not the issue.
    I know it is doing it and how.

    I want to setup a purely transcoding vm where I could drop a file and get the mp4 one back without need of running my desktop for hours if I have a few movies to convert.

    Really looking into handbreak automation script setup I found on Google. Seams interesting if I can manage to set it up. I was hoping for something with at least WebUI to use.
     
    #8
Similar Threads: interesting challenge
Forum Title Date
Linux Admins, Storage and Virtualization Interesting Proxmox Disk Usage Nov 12, 2017

Share This Page