First server build. Need suggestions on chosen hardware.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
- RAM errors are a statistical fact.
20 years ago, you have had 1 MB memory. If you increase RAM by a factor 1000, the chance of memory errors scale the same factor. The more memory the more is ECC important.

- Not being aware of RAM problems does not mean that there are no errors. How many Windows crashes are related to Microsoft bugs, driver problems or RAM problems?

- ZFS does not hold filesystems in RAM. It use RAM as readcache. But unlike some years ago this is a feature of every modern OS to improve performance. A false checksum calculation in RAM may add some risk but you must count against the detection of silent disk errors that are otherwise not detected.

So business data on modern servers with a large RAM without ECC is reckless. The same to filesystems. Copy on write filesystems with versioning and their crash resistent behaviour with data checksums is a huge step forward over oldstyle filesystems and absolutely needed with Terabyte storage. Does not matter if you use btrfs, ReFS or the current top choice ZFS. While missing ECC is only a problem initiated by Intel to earn more money as this is an artificial limitation on cheaper systems, the latter is the default in some years.
 
  • Like
Reactions: wildchild

abstractalgebra

Active Member
Dec 3, 2013
182
26
28
MA, USA
- RAM errors are a statistical fact.
20 years ago, you have had 1 MB memory. If you increase RAM by a factor 1000, the chance of memory errors scale the same factor. The more memory the more is ECC important.

- Not being aware of RAM problems does not mean that there are no errors. How many Windows crashes are related to Microsoft bugs, driver problems or RAM problems?
Gea is spot on, ECC RAM is essential for servers and even critical business desktops (what is the total cost of diagnosing the problem, restoring backups, and all lost productivity).

GOOGLE DRAM Errors in the Wild: A Large-Scale Field Study

Excerpt:
... we observe DRAM error rates that are orders of magnitude higher than previously reported, with 25,000 to 70,000 errors per billion device hours per Mbit and more than 8% of DIMMs affected by errors per year.
...
In many production environments, including ours, a single uncorrectable error is considered serious enough to replace the dual in-line memory module (DIMM) that caused it. Memory errors are costly in terms of the system failures they cause and the repair costs associated with them. In production sites running large-scale systems, memory component replacements rank near the top of component replacements [20] and memory errors are one of the most common hardware problems to lead to machine crashes [19]

Google reports Systems with one ECC corrected error are 13x to 90x more likely to see another correctable error or worse in the next month. i.e. Log and replace the offending DIMM
 
Last edited:
  • Like
Reactions: NetWise

Shiouen

New Member
Sep 17, 2015
17
0
1
30
Thanks again for all the replies guys, really appreciate it!

I've been searching for other options than the c2550 and finally decided giving µATX a chance too after looking at the Node 804. Still, same problems/annoyances persist. So many server boards seem totally overpriced for the older-tech features they provide. €200 euros and up for a simple feature board that still uses SATA II ports raises so many questions in my head..


I have finally found a µATX that should work for me: C226M WS
-- can I use the sata connectors with VT-d?
I still dislike the price though. I've found a mITX board: E3C226D2I
Still dislike the price. :p

I've first looked at many other options, even supermicrosystem boards. Sadly the latter is hard to come by where I live for some reason and is again, expensive. I'm getting the feeling here that I've missed my boat and this is a time of transition towards skylake.


Talking about skylake.. DDR4 would enable me to use 32GB of RAM on a mITX, right?
  • Is it worth the wait?
  • Will I eventually spare some money/effort in short amount of time?
  • Will the features provided on these boards be more streamlined? (e.g. no SATA II and III mixed together)
  • How long would it take for skylake to be at an affordable price point?

Skylake aside, I still have splitting off the NAS from my server as an option. NAS is what I currently need most, the rest can wait later.
I just feel so unsatisfied about the availability, prices and options.. I just keep running circles again haha.
 
Last edited:

archangel.dmitry

Active Member
Sep 11, 2015
224
40
28
US
You should be able to use VT-D with the board.

You actually have 2 choices, wait for Skylake and a) get Skylake-based build with DDR4 support b) get Haswell-based build for less money once people start switching to Skylake.
 
  • Like
Reactions: Shiouen

Solkku

Member
Jul 31, 2015
88
10
8
43
You're process of thinking is very similar to what I went through... Starting small (Avaton ITX board), then realizing it doesn't do the job and move one step bigger in size to mATX. This is what I ended up with after the same thinking/planning process. :) Fingers crossed the mobo comes next week, so I can start building and testing.
 
  • Like
Reactions: Shiouen

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
The first to use new hardware or software is the first to find problems, bugs or incompatibilities.
Mostly I prefer profen and stable configs - no surprise.
 

abstractalgebra

Active Member
Dec 3, 2013
182
26
28
MA, USA
Thanks again for all the replies guys, really appreciate it!

I've been searching for other options than the c2550 and finally decided giving µATX a chance too after looking at the Node 804. Still, same problems/annoyances persist. So many server boards seem totally overpriced for the older-tech features they provide. €200 euros and up for a simple feature board that still uses SATA II ports raises so many questions in my head..


I have finally found a µATX that should work for me: C226M WS
-- can I use the sata connectors with VT-d?
I still dislike the price though. I've found a mITX board: E3C226D2I
Still dislike the price. :p

I've first looked at many other options, even supermicrosystem boards. Sadly the latter is hard to come by where I live for some reason and is again, expensive. I'm getting the feeling here that I've missed my boat and this is a time of transition towards skylake.
For flexibility mATX gives you a lot more choices going forward. For example, you can add more Network Cards including really cheap Infiniband Cards for ~$30, you can get 10GB or 40/56GB for a bit more. That is really fast vMotion and file transfers. Or add a PCIe SSD Drive to get 4-10x more IOPs speed for your VMs.

I'm using the ESXI Napp-it All-in-one and it has been both speedy and rock solid reliable doing VT-d with a SAS2008 controller (Dell H310 or M1015 or ...). I use the onboard sata for extra drives for backup and SSDs. Asrock does have some nice motherboards.
 
Last edited:
  • Like
Reactions: Shiouen

canta

Well-Known Member
Nov 26, 2014
1,012
216
63
43
go with matx or full atx or extended atx for flexibility.

mini-itx is limited for expansion.
 

Zathras

New Member
Sep 15, 2015
16
12
3
Ottawa, Canada
Just to give you my hardware experience, I built my NAS in the Node 804 using a Supermicro X10SLL-F uATX board and I think it's a perfect match. CPU for now is a Haswell Pentium (they still have ECC support) while I decide if I want to go to a Xeon E3 with vT-D, which would be a drop-in replacement. I got 16GB of ECC RAM to start since it was only ~$10 more than non-ECC, with the option of 32GB in future. Using the onboard SATA plus an LSI 9211-8i I can have as many hard drives as I can fit, and I still have spare PCIe slots for high speed network or external storage if I want.

I chose this setup over the Avoton solutions because I wanted greater flexibility, and those boards were pretty expensive in comparison.
 
  • Like
Reactions: Shiouen

Shiouen

New Member
Sep 17, 2015
17
0
1
30
Just to give you my hardware experience, I built my NAS in the Node 804 using a Supermicro X10SLL-F uATX board and I think it's a perfect match. CPU for now is a Haswell Pentium (they still have ECC support) while I decide if I want to go to a Xeon E3 with vT-D, which would be a drop-in replacement. I got 16GB of ECC RAM to start since it was only ~$10 more than non-ECC, with the option of 32GB in future. Using the onboard SATA plus an LSI 9211-8i I can have as many hard drives as I can fit, and I still have spare PCIe slots for high speed network or external storage if I want.

I chose this setup over the Avoton solutions because I wanted greater flexibility, and those boards were pretty expensive in comparison.
Thanks for the input, it totally helps! I have the feeling that I'm getting used to the idea of waiting for Skylake. If I were able to get supermicro boards easily for cheap I'd be sold. Sadly I am not able to where I live (Belgium).

I'm not very sure when I finally find the parts I want, but when I do I'll definitely post a build log! Wouldn't have gotten anywhere without this thread and all the advice :).

But yeah currently it seems Skylake attracts me a lot since it might be available around the end of the year with an affordable price tag. Can't wait for news about Skylake releases.
 
Last edited:

Solkku

Member
Jul 31, 2015
88
10
8
43
If I were able to get supermicrosystem boards easily for cheap I'd be sold. Sadly I am not able to where I live (Belgium).
Are you just not able to find them there or are the prices just sky-high? That seems really strange to me, I mean their main EU-warehouse is in the Netherlands, like right next door to you. o_O You'd think it's not a matter of high shipping costs at least.
 

Zathras

New Member
Sep 15, 2015
16
12
3
Ottawa, Canada
If I were able to get supermicrocomputer boards easily for cheap I'd be sold. Sadly I am not able to where I live (Belgium).
I definitely found it tricky to buy Supermicro stuff as a regular consumer in Canada, but it was definitely worth it. Their sales are almost entirely B2B and large accounts, so you are expected to have a sales rep you can call with any questions.

That said, they list their local distributors on their site, so you could check out the Belgian reseller (Ahead-IT - Supermicro Servers - Buy and configure online once you're looking to buy. That motherboard I got sells for €173 for example, which doesn't seem too bad.
 

Shiouen

New Member
Sep 17, 2015
17
0
1
30
Are you just not able to find them there or are the prices just sky-high? That seems really strange to me, I mean their main EU-warehouse is in the Netherlands, like right next door to you. o_O You'd think it's not a matter of high shipping costs at least.
Prices are quite high. €250-400 for most boards for E3v3's. Most E3-1231v3 I can find (new ones, no ebay and the likes) end up costing about €270-325. I guess it's wishful thinking that waiting for skylake will save me some money haha.

I definitely found it tricky to buy Supermicro stuff as a regular consumer in Canada, but it was definitely worth it. Their sales are almost entirely B2B and large accounts, so you are expected to have a sales rep you can call with any questions.

That said, they list their local distributors on their site, so you could check out the Belgian reseller (Ahead-IT - Supermicro Servers - Buy and configure online once you're looking to buy. That motherboard I got sells for €173 for example, which doesn't seem too bad.
I could use my business (activities in all things computers/development) to get B2B sales. Problem is that they mostly expect me to buy higher quantities of one item I think. I also have used Ahead-IT to search for a couple of motherboards, but yours drops like €50 under all the rest! Def going to check that out! :)
 

Solkku

Member
Jul 31, 2015
88
10
8
43
One option would be to wait for Skylake and then grab some deals on Haswell E3-1200v3 series CPU & motherboard & DDR3 ECC from early adopters. :) That is if they ever arise, don't know if people will eagerly update from Haswell to Skylake.

Edit: I guess 270€ is quite a normal price for a new E3-1231v3... The lowest on Amazon.de is ~260€ atm. I managed to snag a deal of a boxed one at Mindfactory.de at 247,xx€ (still seems to be on sale). But of course had to order some other stuff for the new server from there too to justify the 19,99€ shipping costs to Finland. :)
 
  • Like
Reactions: Shiouen

Shiouen

New Member
Sep 17, 2015
17
0
1
30
One option would be to wait for Skylake and then grab some deals on Haswell E3-1200v3 series CPU & motherboard & DDR3 ECC from early adopters. :) That is if they ever arise, don't know if people will eagerly update from Haswell to Skylake.

Edit: I guess 270€ is quite a normal price for a new E3-1231v3... The lowest on Amazon.de is ~260€ atm. I managed to snag a deal of a boxed one at Mindfactory.de at 247,xx€ (still seems to be on sale). But of course had to order some other stuff for the new server from there too to justify the 19,99€ shipping costs to Finland. :)
I now remember why I wasn't happy about the Ahead-it Webshop: Prices shown are VAT excl., which is 21% in Belgium. They sell your motherboard for €253.25. The prices a friend of yours gave you are pretty nice..

Btw, your motherboard supports Intel VHD. Do you actually use it?
 

Solkku

Member
Jul 31, 2015
88
10
8
43
I see you took a look at the mini spreadsheet I made. :) Yeah, he's a "high-tier suit" of a Scandinavian Supermicro distributor, so I guess that helped with keeping the price low for me... free shipping also, which is always a bonus. I would probably never have bought it at normal price. That 222€ price is incl. Finnish VAT 24%, it was around 179€ excl. VAT.

Actually I haven't even looked what Intel VHD does yet, it was just a small difference I spotted between this an the other two, so I put it in the spreadsheet.
 
  • Like
Reactions: Shiouen