PMS 4.0...PMS 5.0...PMS 6.0...No PMS 7.0! Plex/Storage server upgrade [PICS]

Ixian

Member
Oct 26, 2018
64
9
8
Also don't forget you have a lot of options on the secondary market for E5 v3's right now. For example the E5-2660V3 is the same speed (actually base clock is 100mhz higher) as the 2680, but with 2 less cores (10 vs. 12). It's also got a 105w TDP vs. 120w. And you can find them on Ebay for around $230.

If you don't absolutely need the extra 2 cores - and if you needed 12 cores instead of 10 I imagine you'd know it - that's an option that would save you some money and power/thermal output.

The max turbo on both is 3.3GHz, though you probably already know you'll never see that in a multi-use environment like Unraid since it depends on all but one core being idle. The real turbo speed in my experience is a hair over 3GHz, for all cores.

Emby seems to do a better job, in my experience, than Plex when it comes to scaling across cores. They both use ffmpeg but they also both modify it to their own needs and use different builds. I still run a Plex server because my parents are shared with it and I am not mentally ready to walk them through switching yet so this is something I've noticed over the last year.

My Emby server can push 4 1080p streams without breaking a sweat and of course Direct Play uses very little CPU so I can do that too. This is running inside a Debian VM with docker that has 6 of my 12 cores (physical cores) assigned to it.
 

stresslvl0

New Member
Jan 7, 2019
4
1
3
Few reasons.

1. When I do upgrades I don't like going with previous generation hardware even though I know it's often a better value. When I upgrade I like to go with recent hardware if not cutting edge so that I don't feel the need to upgrade it again in a year.
2. TDP. I don't want to go for more than an 85w TDP CPU because this server sits in a closet. The 2680v3 is 45w higher.
3. I don't see any MicroATX E5v3 boards with dual SFP+ and onboard HBA that supports 10+ drives.

P.S. I haven't actually bought the Xeon Gold 5115 yet. I'm still considering the Xeon Silver 4114 for $670.
I'm the same way re: #1 and #2. To keep TDP and idle power consumption way down I'm thinking the 4109T is going to be the way to go for my build. It's only 70W TDP compared to all the others at 85W. It costs the same as the 4110, but the efficiency is worth the slight speed difference. I think the speed will be fast enough and then when I need to do more transcoding the Nvidia card will be a good fit.

My UPS is showing 110w with it idle right now. My drives are spun down (I imagine you'll do the same with Unraid, since it is easier to keep drives spun down with a NSA). Figure 170-190w under load, if I remember. That's going off my UPS though - the server is the only thing hooked up to it at the moment, but I don't know what overhead it adds/how accurate it is.

[...]

Edit: Note I am using the 10TB Reds, which are known for lower power consumption (around 7w spun up). That's well over a third lower than the 8TB version, and of course faster drives (7200 Golds, etc.) will consume even more. With 8 of them that will have a big impact on the totals.
This is 110W idle with 8x 10TB WD Reds spun down and 3 SSDs? The Reds are listed as 2.8W idle, so that's 87W idle not counting drives? That seems a little high, no?
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
Hey guys, been MIA the past 2 weeks as I've been taking care of a newborn ;). In the few moments I've had to think about anything other than a new baby in the past few weeks I've decided to go with @Ixian 's suggestion and do an E5-2680v3 build. The fact is I would have been paying a huge premium for the Xeon Scalable CPU's and the Xeon E-2100 just isn't well rounded enough of a platform for me to use as my single all-in-one home server.

I'm re-using the following items from my current Xeon D-1541 build:

  • Node 804 Case
  • 8 x 10TB WD Golds
  • 960GB Micron 5100 Pro M.2 SSD
  • 64GB (32GB x 2) DDR4-2133 RDIMMs
  • PSU

New parts for build:

  • SuperMicro X10SRM-F (purchasing this week)
  • Intel E5-2680v3 (purchased off eBay for $275)
  • Intel X520-DA2 (found BNIB in storage...JACKPOT)
  • Intel Optane 280GB 2.5" SSDs x2 (purchased)
  • Supermicro AOC-SLG3-2M2 PCIe AOC (purchased)
  • Noctua NH-D9DX i4 3U CPU Cooler

@Ixian Can you confirm the linked Noctua cooler I listed is the right one that you referenced?
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
So I just realized the PSU in my current server is only 360w so that's probably not going to do for the E5-2680v3. Think 500w will suffice?
 

maze

Active Member
Apr 27, 2013
556
84
28
Good question..

2x 2680, 2dimms, mobo, 8x drives, 10gbase-t, ssd/nvme’s, fans and u want to keep 50w clear for the quadro card Down the Line?

I’d say 600 to be sure, depends what kind of max % load your aiming for?
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
Good question..

2x 2680, 2dimms, mobo, 8x drives, 10gbase-t, ssd/nvme’s, fans and u want to keep 50w clear for the quadro card Down the Line?

I’d say 600 to be sure, depends what kind of max % load your aiming for?
The board I'm using is single CPU but yes want some headroom to add Quadro card down the line as an option. I want to be able to support 100% load on the CPU.
 

maze

Active Member
Apr 27, 2013
556
84
28
The board I'm using is single CPU but yes want some headroom to add Quadro card down the line as an option. I want to be able to support 100% load on the CPU.
sorry, should have specified. PSU load % was what i ment.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
sorry, should have specified. PSU load % was what i ment.
Oh gotcha. Well I like keeping my server as quiet as possible so keep the load low enough so the fan isn't really needed is the way I usually go.
 

maze

Active Member
Apr 27, 2013
556
84
28
Should be good then yes.

My point on the load on the psu being that most of Them his optimal efficiency at 80-90% load - but if your power is cheap enough.. who cares :)
 

Ixian

Member
Oct 26, 2018
64
9
8
That's the correct cooler and a good Gold rated 600-650w PSU should serve you well, not much of a price difference between them and, say, 500w either, for most brands.

The 804 actually has a decent amount of room, and built-in velcro straps, right behind the PSU making it a good place for cable stowage (it doesn't block any fans/vents there either). So a modular PSU is purely up to you.
 

Ixian

Member
Oct 26, 2018
64
9
8
Here's a pic of that side on mine:

IMG_2571-2 - Copy.jpg

Not as neat as it could be, but the important thing is the cables are all accessible (i.e. not twisted around one another) and aren't blocking airflow, so I don't really care what it looks like.

Also note I labeled all my drives; I *highly* recommend doing this. Will save you a lot of time and grief if you ever need to replace one. I used a $15 Dynamo label maker but masking tape and a sharpie would work just as well, just needs to be legible of course.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
Here's a pic of that side on mine:

View attachment 10330

Not as neat as it could be, but the important thing is the cables are all accessible (i.e. not twisted around one another) and aren't blocking airflow, so I don't really care what it looks like.

Also note I labeled all my drives; I *highly* recommend doing this. Will save you a lot of time and grief if you ever need to replace one. I used a $15 Dynamo label maker but masking tape and a sharpie would work just as well, just needs to be legible of course.
Tha js for the suggestion. I actually have been planning to label the drives for a few months now and since I'll be reworking this build it's the perfect opportunity.

The last item(s) I'm looking for are some decent write endurance rated 500GB-1TB 2.5" SSDs to throw behind the front cover of the node. Trying not to spend more than $300-350ish in total on the 2.
 

Ixian

Member
Oct 26, 2018
64
9
8
Tha js for the suggestion. I actually have been planning to label the drives for a few months now and since I'll be reworking this build it's the perfect opportunity.

The last item(s) I'm looking for are some decent write endurance rated 500GB-1TB 2.5" SSDs to throw behind the front cover of the node. Trying not to spend more than $300-350ish in total on the 2.
What are you doing with them? If it's for run of the mill media server dockers, jails, vms, etc. most decent consumer drives will provide "good enough" write endurance. I use Samsung 860 Evos. At my current rate I won't write out data past the endurance benchmark before the warranty expires and then some. Heck, my backup Freenas box is still rocking a single 500GB Toshiba that is 4+ years old and doing fine (though naturally I replicate snapshots of it to my main array frequently in case it one day decides to go). For most of those 4 years it was in daily use as my Plex jail/VM storage/etc. volume.

If it's for a SLOG or heavy-use db or other high write scenario I like the Optane 900p series. You can also find Optane S3700's cheap occasionally on Ebay, which have even higher endurance. I personally use a DC P3700 (NVMe version) in one server and the 905p in the other, for SLOG and application cache duties.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
What are you doing with them? If it's for run of the mill media server dockers, jails, vms, etc. most decent consumer drives will provide "good enough" write endurance. I use Samsung 860 Evos. At my current rate I won't write out data past the endurance benchmark before the warranty expires and then some. Heck, my backup Freenas box is still rocking a single 500GB Toshiba that is 4+ years old and doing fine (though naturally I replicate snapshots of it to my main array frequently in case it one day decides to go). For most of those 4 years it was in daily use as my Plex jail/VM storage/etc. volume.

If it's for a SLOG or heavy-use db or other high write scenario I like the Optane 900p series. You can also find Optane S3700's cheap occasionally on Ebay, which have even higher endurance. I personally use a DC P3700 (NVMe version) in one server and the 905p in the other, for SLOG and application cache duties.
They will be used for less important VMs and dockers but the big hit on them is downloads. I download a LOT of NZBs and every single one of them has to be extracted so I see a ton of writes on my download directory.
 

Ixian

Member
Oct 26, 2018
64
9
8
Not as much as you think. That 500GB Toshiba had SABnzbd on it too.

I mean, I guess it depends on your definition of "a lot" but....

Consider also, from a budget perspective, that given you have two additional drive mounts below the motherboard - where a 2.5 SSD wouldn't even remotely be in the way - you might want to just get a smaller, cheaper SSD and use that as the dedicated NZB download/extraction storage point (while keeping the app itself on your "primary" SSDs), and if it dies in a couple years, so what, super easy to replace, you don't have much at risk, and drives will be even cheaper.

Vs. investing in a much more expensive high endurance drive(s) today. There are certainly good reasons to do that, but "NZB downloading" isn't really one of them :)
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
Not as much as you think. That 500GB Toshiba had SABnzbd on it too.

I mean, I guess it depends on your definition of "a lot" but....

Consider also, from a budget perspective, that given you have two additional drive mounts below the motherboard - where a 2.5 SSD wouldn't even remotely be in the way - you might want to just get a smaller, cheaper SSD and use that as the dedicated NZB download/extraction storage point (while keeping the app itself on your "primary" SSDs), and if it dies in a couple years, so what, super easy to replace, you don't have much at risk, and drives will be even cheaper.

Vs. investing in a much more expensive high endurance drive(s) today. There are certainly good reasons to do that, but "NZB downloading" isn't really one of them :)
Another good point.

On second thought I forgot I have a Micron 5100 Pro 960GB M.2 drive I can use for downloads/DVR. It has 3.4PBW endurance. Then I can use a pair of lower capacity 860 EVOs for my lower performance yet still highly used (thus the redundancy) data workloads.
 

Ixian

Member
Oct 26, 2018
64
9
8
Yeah that works, and I do virtually the same thing. I carved up my 400GB Optanes for SLOG duties (10GB each volume, not much) and a couple hundred GB for caching purposes (NZB, Emby temp files, and so on). Super fast and high endurance. If I wasn't using a SLOG I'd have just gone with something cheaper though.

I over-provisioned the remaining space with Intel's tool to improve wear-leveling. I think you can only do that with their data center drives though, and it's a bit overkill.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,279
548
113
Got some of the new parts ready to go. Waiting on motherboard and PSU which should be here Friday. I'll post some pre and post upgrade pics of my Node build once I start to tear things down.