Updated: Krypton Plex Server - Open Compute Node - HGST 4U60 added

[Gene]

Build’s Name: Krypton - Plex Server Updated
Operating System/ Storage Platform: Debian 64 bit - OpenMediaVault 3.0
CPU: Dual E5-2650L (for power savings) originally Dual E5-2670
Motherboard: Quanta Windmill Open Compute Node
Storage Chassis: HGST 4U60 G1 version
Drives: sytem raid 1 Intel 3500 80gb ssd system drive x2 - Spinning rust uses Snapraid and mergerfs Storage is mixed and match drives mostly 8TB. Total storage well over 100TB. 26 slots used in hgst 4u60
RAM: 16GB (8gbx2) Nanya 2Rx4 PC3-10600R ECC x2
Add-in Cards: Mellanox Mezzanine card CX341A 10gbit SFP+, LSI Sas 9300-8E, old: Sas Dell H3-25553-01A (burned itself up from overheating)
Power Supply: Built in 208volt ps for opencompute nodes, HGST 2x 1.65Kw psus only 1 plugged in for efficiency gains
Other Bits: Racked in HP 48U, 2x 8644 to QSFP+ cables UPS: Eaton 5px3000i outputting 208 volt, Eaton EBM 72v battery module, APC 9629 step-down transformer

Usage Profile: Openmediavault on debian x64 - running SABNZBD plus, Couchpotato, Plex, Emby (mainly for livetv with iptv/antenna), Headphones, SickRage, Sonarr, transmission, deluge

Power usage: Total idle power draw for rack: 377w - breaking it out HGST with 26 drives - 150W, Open Compute Node 1 with Lsi 9300-8E sas card and mezzanine 10gbit card - 80W, pfsense router - 27w (3x gig ports, 2x10gbit sfp+) - UPS system: Eaton 5px3000i 208v output, Eaton 72V extra battery module, plus APC 9626 step down transformer to 120v - 81 watts idle - All other power is to misc. network switch/cable modem/wifi access point/av gear 27 watts - Mikrotik CSS326-24G-2S+RM 12 watts (lower power 10 gig switch), Quanta LB4M - 60 watts (retired), Tivo recording combo x2 20 watts each (retired instead use iptv and emby now)

Todo: Cable management, re-install rack sides/doors, clean up basement area

Full listing of everything in rack and media setup in general

 

[Patrick]

Gene - looks very cool! On the Open Compute nodes, how are you securing them into the racks?

 

[Gene]

Gene - looks very cool! On the Open Compute nodes, how are you securing them into the racks?

Well,.right now it is sitting on the SC846 with just the posts secured into rack so it can't move front to back. I'm not sure if there is a better way to rack it. It fits between the standard rack posts once you get it in at an angle and is about the same length as the SC846 case. The OC node is not very heavy and my 846 mount should be able to support the weight so I may just leave it as is.. I'm sure i couldn't get away with that if I wasn't in a home environment

 

[Gene]

Just some thoughts.. This was a very cost effective media server build. I should be able to mostly fund it from selling older gen components. Decent power consumption and high performance in a home environment. With that much processing power (39K passmark) it's pretty future proof for a while. It can even handle HEVC/H265 encoding/decoding duties. The 2nd node will not be on very often unless i get heavy transcoding loads from Plex and then i'll get that setup to WOL and integrate with the a modified plex transcoder. Some more upgrades in the near future so I can reuse my three 3in5 sata enclosures plan on getting a Rosewill RSV-L4500 and reusing my intel sas expander. Total 3.5" hard drive slots = 39. In use: 12 - so uh yeah plenty of room to grow haha

 

[Kneelbeforezod]

Did you get the open compute node from Ebay? And why did you go with the node int he first place?

 

[dicecca112]

Gene, are the benefits of having a cache SSD for Plex noticeable? Considering it on my Plex VM.

 

[rubylaser]

Gene, are the benefits of having a cache SSD for Plex noticeable? Considering it on my Plex VM.

Having your Plex database and thumbnails on an SSDS makes browsing the library MUCH faster. Even on huge libraries.

 

[RobertFontaine]

Very nice use of the 2670's. I am quickly discovering that my ability to use a single workstation for everything is going to run out soon. A rack in the next room with a stack of these might solve all my VM and compute issues. A stack of those fans would definitely result in a divorce however.

 

[Gene]

Did you get the open compute node from Ebay? And why did you go with the node int he first place?

Yes ebay but it was a single seller. I was looking for a node with pcie risers included along with the airdivider insert. Mellanox mez card was another ebay seller. As to why, i did my research and i already had a ups that output 208-240v. The oc nodes are the cheapest thing to use the e5-2670s and they are built to be as efficient as possible. I basically doubled my processing ability and lowered my power consumption at very little cost after i sell my old server parts for goong rates

Gene, are the benefits of having a cache SSD for Plex noticeable? Considering it on my
Plex VM.

Yes, very if you have a large db it is very noticable. Also i use it for transcode temp space area

 

[Gene]

Very nice use of the 2670's. I am quickly discovering that my ability to use a single workstation for everything is going to run out soon. A rack in the next room with a stack of these might solve all my VM and compute issues. A stack of those fans would definitely result in a divorce however.

The node isn't terribly loud. Ill get my db sound meter out and get a reading. That apc transformer is definitely the loudest thing in the rack.

 

[RobertFontaine]

Warm them up a little or set fans to 100% in bios....

 

[Kneelbeforezod]

How suitable is OCN as a workstation? Like for video editing with Premiere and Handbrake on W7/10

 

[Gene]

Warm them up a little or set fans to 100% in bios....

Well the basement is cool so I have not had them spool up yet even when trascoding at half load

How suitable is OCN as a workstation? Like for video editing with Premiere and Handbrake on W7/10

I wasn't planning on using it for a dedicated windows environment. It's working good enough for a win7 guest os so i can run wmc for cable tv. I'd redirect your question to this thread as some of them were using it with windows. I do recommend you stick with the quanta brand and not the wiwynn i ended up with as quanta has firmware updates which would be very useful for windows : Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack - I think the biggest problem for graphics is that while they are a pcie 3.0, they are only 8x maybe?

 

[nk215]

Can I ask why so much CPU power for Plex? I have my Plex server running on a v3-1270v2 and everything runs well. Granted, when I ripped by Blurays, I already try to use the mp4 container as much as possible. Transcoding is mostly for remote access to lower resolution.

 

[Gene]

Can I ask why so much CPU power for Plex? I have my Plex server running on a v3-1270v2 and everything runs well. Granted, when I ripped by Blurays, I already try to use the mp4 container as much as possible. Transcoding is mostly for remote access to lower resolution.

I have a lot of family and google fiber should be installing in the next year so i wanted to have plenty of cpu available. Right now i just have about 10-12 active users. Also some of my video content is in h265/hevc so it needs beefy cpu to convert back to h264

 

[zhoulander]

Nice setup! Are you just looping SFF SAS cables out the back of the OCN and into the pci area of the SC846? Any pics of the rear you can show?

 

[Gene]

Nice setup! Are you just looping SFF SAS cables out the back of the OCN and into the pci area of the SC846? Any pics of the rear you can show?

Map of setup for sc846 drives is | sata drives <--> sata 2 backplane <---> 8087 cable x2<--> dual 8087 to 8088 adapter <--> sas 8088 cable x2<--> dual 8087 to 8088 adapter <--> Ibm m1015 sas controller in open compute node |

I'll be simplifying it with an external sas card so I can get rid of the adapter in the OC node. Also you can see my cables are a mess at the moment. Eventually i'll do a cable cleanup

 

[MountainDew]

Having your Plex database and thumbnails on an SSDS makes browsing the library MUCH faster. Even on huge libraries.

I don't disagree that SSDs will be faster compared to spinning drives. However, I saw similar performance moving away from my SSD to using my RAID 6 array which consists of 6Gb SAS drives. I haven't noticed any performance decrease on anything when using Plex. Just some food for thought.

 

[Gene]

I don't disagree that SSDs will be faster compared to spinning drives. However, I saw similar performance moving away from my SSD to using my RAID 6 array which consists of 6Gb SAS drives. I haven't noticed any performance decrease on anything when using Plex. Just some food for thought.

I would agree you wouldn't see a big difference between a raid6 array and the ssd, however I'm running snapraid and drive pooling on the sata drives so no multi-drive performance increase. Also I have it setup to spin drives downs when idle after 2 hours so having any kind of reading/writing on a separate ssd makes more sense for that.

 

[rubylaser]

I don't disagree that SSDs will be faster compared to spinning drives. However, I saw similar performance moving away from my SSD to using my RAID 6 array which consists of 6Gb SAS drives. I haven't noticed any performance decrease on anything when using Plex. Just some food for thought.

I have a huge Plex library (by library I mean the actual Plex library, cache directory, and transcode directory), and having it on a ZFS mirror of (2) 400GB Intel S3700's is a marked improvement for me on my Plex clients (a mix of Roku 3/4's, an Android Mini MX, and a few Celeron 1037u's). I also use this ZFS pool for my other Docker containers.

With the price and reliability of good enterprise SSDs, I see a huge benefit from running all of this stuff on low power enterprise SSDs vs. an array of spinning disks. I also run a large SnapRAID + mergerfs array like @Gene, so keeping as many reads/writes off my spinning disk pool keeps disks spun down more of the time (saving me money in energy costs).