My first server; Supermicro SC846E16-R1200B

[ricktsd]

Hey guys,

So this is the first time I'm building a server like this. I just wanted to run this hardware by you guys and see what you think. I am a complete newbie to this, and want to get it done right the first time.

I plan to use this as a file server, Plex, Kodi, cloud server.

I saw a deal for this in the other subforum, and I'm going to try to get replicate it, even though it seems dead right now.

Chassis: Supermicro SC846E16-R1200B
Power Supply: 2 x PWS-1K21P-1R
Power Distributor: PDB-PT846-8824
Backplane: BPN-SAS2-846EL1

Motherboard: Supermicro MBD-X9SCM-F-O

Amazon.com: SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard: Electronics

RAM - Crucial 16GB x 2

Crucial 32GB Kit (16GBx2) DDR3/DDR3L-1600 MT/s (PC3-12800) DR x4 RDIMM Server Memory CT2K16G3ERSLD4160B / CT2C16G3ERSLD4160B at Amazon.com


RAM - Crucial 8GB x 4

Crucial 16GB Kit (8GBx2) DDR3/DDR3L-1600MT/s (PC3-12800) DR x8 ECC UDIMM Server Memory CT2KIT102472BD160B/CT2CP102472BD160B at Amazon.com

CPU - Intel Xeon E3-1230v2:

Amazon.com: Intel Xeon Quad-Core Processor E3-1230 v2 3.3GHz 8MB LGA 1155 CPU LGA BX80637E31230V2: Computers & Accessories

CPU Cooler - Noctua NH-U9DXi4 (maybe not necessary?)

Amazon.com: Noctua i4 CPU Cooler for Intel Xeon CPU_ LGA2011, 1356 and 1366 Platforms NH-U9DXi4: Computers & Accessories


OS drive - SATADOM 32 GB Internal Solid State Drive:

Amazon.com: SATADOM 32 GB Internal Solid State Drive: Computers & Accessories

RAID card: 1 x ServeRAID M1015

Cables: 1 x SFF-8087 to SFF-8087: Supermicro CBL 0281L 75cm Mini SAS SFF 8087 to Mini SAS SFF 8087 Internal | eBay

I currently have 2x5TB, 6x4TB, 2x3TB, 3x1.5TB drives

My questions if someone could help:

1. Is there any other hardware I need to buy?
2. How many M1015 do I need for 24 hard drives and how many SFF-8087 do I need?
3. Any recommendations on fans to replace the stock ones to keep it fairly quiet?

Probably FreeNAS on it.

 

[voodooFX]

Hi ricktsd and welcome to sth

I had a quick review of your hardware and I see three problems

1. You decided to go for xeon E3 platform (it's ok) but this platform (so also the mb you choosed) does not support ECC Reg. RAM but only ECC Unbuffered/Unregistered. You need to change the RAM

2. 1U power supplies are very loud and you will have two of them..

3. Your backplane connections are SATA so you do not need a Mini-SAS (SFF-8087) to Mini-SAS (SFF-8087) cable but Mini-SAS (SFF-8087) to (4x) SATA

Since the case has 24 drives and you can connect up to 8 of them to a single M1015 you will need 3 cards and 6 cables

 

[ricktsd]

Hi ricktsd and welcome to sth

I had a quick review of your hardware and I see three problems

1. You decided to go for xeon E3 platform (it's ok) but this platform (so also the mb you choosed) does not support ECC Reg. RAM but only ECC Unbuffered/Unregistered. You need to change the RAM

2. 1U power supplies are very loud and you will have two of them..

3. Your backplane connections are SATA so you do not need a Mini-SAS (SFF-8087) to Mini-SAS (SFF-8087) cable but Mini-SAS (SFF-8087) to (4x) SATA

Since the case has 24 drives and you can connect up to 8 of them to a single M1015 you will need 3 cards and 6 cables

wow, thanks!

1. Is this ram ok? 2 packs for total of 32GB. Crucial 16GB Kit (8GBx2) DDR3/DDR3L-1600MT/s (PC3-12800) DR x8 ECC UDIMM Server Memory CT2KIT102472BD160B/CT2CP102472BD160B at Amazon.com

2. Noise is ok; would prefer it to be quieter, but not a real problem if it's loud

3. This one? Amazon.com: Discrete SATA to SFF-8087 Mini SAS Reverse breakout cable: Industrial & Scientific
The picture of the backside on this page doesn't show anywhere to connect the SATA cables though BPN-SAS2-846EL1 SAS Expander Backplane - Thomas-Krenn-Wiki

 

[voodooFX]

Yes this RAM is correct.
About the backplane I was looking at the wrong picture, you were right you need Mini-SAS to Mini-SAS

EDIT: If you will go for FreeNAS I would suggest you two drives in mirror for boot/os

 

[ricktsd]

given the hard drives i already have, what would be the best software option? seems like freenas wont utilize all of the space well when the drives are all different sizes

 

[HotFix]

My questions if someone could help:

1. Is there any other hardware I need to buy?
2. How many M1015 do I need for 24 hard drives and how many SFF-8087 do I need?
3. Any recommendations on fans to replace the stock ones to keep it fairly quiet?

Probably FreeNAS on it.

#1. It looks like voodooFX has you covered here.
#2. The M1015 should be able to handle 24 drives no problem as long as you flash it to IT mode. One person is currently running it with 48 drives, and it should be able to handle a lot more as reported here: https://forums.servethehome.com/index.php?threads/need-help-with-home-storage-server.4237/
#3. There is general SilentPCReview write-up on quiet fans here: Recommended Fans | silentpcreview.com
Then there is this page I found where some anal retentive people like myself did some actual spread sheet analysis of fans and found the SwiF2 fans were the best out of a combination of tests:
The 5X3 Cage review - Norco, SuperMicro, iStarUSA and Icy Dock
My only complaint with a lot of these fan reviews/recommendations is they tell you what initial testing results they had, not how the fans sound in a year of constant 7x24 operations. I have bought a number of recommended "quiet" fans over the year only to have them end up being the loudest part of my servers after a year or so of constant run time.
However I have been told if you plug the fans in the SC846 into a Supermicro motherboard's PWN connectors that it will do a decent job of running them only as loud as it needs to, so I recommend you try that route first before spending more money potentially unnecessarily.

 

[ricktsd]

Yes this RAM is correct.
About the backplane I was looking at the wrong picture, you were right you need Mini-SAS to Mini-SAS

EDIT: If you will go for FreeNAS I would suggest you two drives in mirror for boot/os

do i still need a total of 6 cables? sorry for the stupid question, but i've never dealt with SAS before, so i'm not sure how it all works.

I see each M1015 has 2 ports, but there are three ports on the backplane. so how do you connect 3 M1015's to the backplane?

 

[ricktsd]

#1. It looks like voodooFX has you covered here.
#2. The M1015 should be able to handle 24 drives no problem as long as you flash it to IT mode. One person is currently running it with 48 drives, and it should be able to handle a lot more as reported here: https://forums.servethehome.com/index.php?threads/need-help-with-home-storage-server.4237/
#3. There is general SilentPCReview write-up on quiet fans here: Recommended Fans | silentpcreview.com
Then there is this page I found where some anal retentive people like myself did some actual spread sheet analysis of fans and found the SwiF2 fans were the best out of a combination of tests:
The 5X3 Cage review - Norco, SuperMicro, iStarUSA and Icy Dock
My only complaint with a lot of these fan reviews/recommendations is they tell you what initial testing results they had, not how the fans sound in a year of constant 7x24 operations. I have bought a number of recommended "quiet" fans over the year only to have them end up being the loudest part of my servers after a year or so of constant run time.
However I have been told if you plug the fans in the SC846 into a Supermicro motherboard's PWN connectors that it will do a decent job of running them only as loud as it needs to, so I recommend you try that route first before spending more money potentially unnecessarily.

i've been reading a bit, and i haven't come across anything about plugging fans into PWN connectors. i'll give that a shot first instead of buying fans then.

 

[neo]

May I ask why you picked out a Power Distributor and Power Supplies separately?

95% of the SC846 cases on eBay comes with both of your picks as standard hardware options already included. Additionally, those PSUs are loud , get the platinum versions - especially if you are specifying PSUs separately.

 

[ricktsd]

May I ask why you picked out a Power Distributor and Power Supplies separately?

95% of the SC846 cases on eBay comes with both of your picks as standard hardware options already included. Additionally, those PSUs are loud , get the platinum versions - especially if you are specifying PSUs separately.

I only listed them so that it was understood I have a psu present. It should come with the chassis standard like you said.

 

[HotFix]

do i still need a total of 6 cables? sorry for the stupid question, but i've never dealt with SAS before, so i'm not sure how it all works.

I see each M1015 has 2 ports, but there are three ports on the backplane. so how do you connect 3 M1015's to the backplane?

Take 2 since my year and 1/2 old daughter "helped" me delete my original attempt at a response.

Take a look at the thread I started over in the Enclosure's forum where I worked through similar questions:
https://forums.servethehome.com/ind...-for-advice-on-a-case-for-ssds-and-hdds.5840/
Essentially the backplane you have has a single inbound SAS port on it, and 2 outbound ports so you can daisy chain 2 additional enclosures off it (it's in the manual on Supermicro's website which I recommend skimming through - especially appendix D and E where it gives graphical representations of what I am talking about).
The only downside to this approach is you are "limited" to 4 x 6Gbs = 24Gb/s because you are using a SAS2 backplane (versus 4 x 3Gbs with a SAS1 backplane), but this "wide" SAS connection will probably not be saturated by anything you are doing with mostly HDDs. In fact if you don't end up using SSDs you will probably never come close to hitting that limit, unless you daisy chain 2 additional enclosures off the enclosure (which you should use the other port on your 1015 to connect to an external enclosure first as that is a whole additional 24Gb/s channel).

I hope that made sense.

 

[andrewbedia]

2. 1U power supplies are very loud and you will have two of them..

My HP common slot 80PLUS Platinum 460W would like a word with you.

 

[ricktsd]

Since the case has 24 drives and you can connect up to 8 of them to a single M1015 you will need 3 cards and 6 cables

sorry, something isn't clear to me.

how do you connect the M1015 to 8 drives? each M1015 has two ports, and the backplane only has a total of 3 ports. how do you know which port connects to which other port using the mini-sas to mini-sas cable?

 

[Scott Laird]

Read the manual: http://www.supermicro.com/manuals/other/bpn-sas2-846el.pdf. Specifically page 3-5. You should be able to use 1 or 2 cables, and it'll Just Work. 2 should give you more bandwidth, if that's an issue for you.

 

[ricktsd]

Read the manual: http://www.supermicro.com/manuals/other/bpn-sas2-846el.pdf. Specifically page 3-5. You should be able to use 1 or 2 cables, and it'll Just Work. 2 should give you more bandwidth, if that's an issue for you.

im just trying to figure out all this, since i was told earlier that I needed 6 cables, and now you're saying i only need one or two. each M1015 is supposed to be able to handle only 8 drives, so if i put 24 in, how would one cable be able to control all 24 drives?

i feel like im coming across like an idiot, but this concept is strange to me. ive been googling for pictures or video of this chassis and HBA combination with 24 drives and can't find anyone with the same configuration.

EDIT: ok, after some more googling, it seems you only need one M1015 and one SFF-8087 cable to drive all 24 drives.

 

[Scott Laird]

The "E16" part of the Supermicro case's model number means that it includes a non-redundant SAS expander backplane. That means that it *has* to talk to a SAS card, not a SATA card, but it's able to talk to either SAS or SATA drives. This is probably the right backplane/case version for you; the alternatives are the A/TQ versions, which simply have 24 SATA ports on the back of the backplane (possibly grouped into denser cables); the E26 version, which is wired for redundant SAS drives; and the B* versions, which do 12G SAS.

SAS expanders are sort of like switches for SAS traffic. You can plug a bunch of drives in one side and one or more SAS cards into the other, and they'll make everything work out. I have a Supermicro 847-E16 case (the one with 12 more drives on the back, on a second backplane), and I only have 1 cable going to each backplane, but I *think* adding 2 cables will Just Work. Worst case, your OS will need to support SAS multipath, which pretty much any OS you'd use for a file server should support. On the other hand, each cable is 4x600 MB/sec, and that's quite a bit of disk traffic. I don't think my ZFS box ever pushes more than ~1.5 GB/sec over SAS in aggregate, even during rebuilds.

Short version: get the right cable, plug the backplane into the M1015, everything'll work like magic, including hot plugging drives.

 

[Scott Laird]

Oh, to clarify: the M1015 isn't limited to 8 *drives*, it just has 8 physical ports. Each one can be used for SAS or SATA. If you get a breakout cable, you could wire them directly to 8 SATA drives, and that'd work just fine. Or you could wire some of the ports into a SAS expander, and then it'll let you plug in more drives. The "8" is just a physical port limit, not any sort of hard cap on what is possible with an expander. You *might* hit a limit at 127 drives, but I don't know of anyone that's cascaded expanders to try to test that with a single card.

 

[T_Minus]

You were told that earlier, but the post was edited, and it has been quoted since the edit.

I think you may have missed that part It was a mistake by the poster originally stating that he saw the wrong image.

You only need 1 HBA and & cable.

 

[neo]

The "E16" part of the Supermicro case's model number means that it includes a non-redundant SAS expander backplane. That means that it *has* to talk to a SAS card, not a SATA card, but it's able to talk to either SAS or SATA drives. This is probably the right backplane/case version for you; the alternatives are the A/TQ versions, which simply have 24 SATA ports on the back of the backplane (possibly grouped into denser cables); the E26 version, which is wired for redundant SAS drives; and the B* versions, which do 12G SAS.

SAS expanders are sort of like switches for SAS traffic. You can plug a bunch of drives in one side and one or more SAS cards into the other, and they'll make everything work out. I have a Supermicro 847-E16 case (the one with 12 more drives on the back, on a second backplane), and I only have 1 cable going to each backplane, but I *think* adding 2 cables will Just Work. Worst case, your OS will need to support SAS multipath, which pretty much any OS you'd use for a file server should support. On the other hand, each cable is 4x600 MB/sec, and that's quite a bit of disk traffic. I don't think my ZFS box ever pushes more than ~1.5 GB/sec over SAS in aggregate, even during rebuilds.

Short version: get the right cable, plug the backplane into the M1015, everything'll work like magic, including hot plugging drives.

Oh, to clarify: the M1015 isn't limited to 8 *drives*, it just has 8 physical ports. Each one can be used for SAS or SATA. If you get a breakout cable, you could wire them directly to 8 SATA drives, and that'd work just fine. Or you could wire some of the ports into a SAS expander, and then it'll let you plug in more drives. The "8" is just a physical port limit, not any sort of hard cap on what is possible with an expander. You *might* hit a limit at 127 drives, but I don't know of anyone that's cascaded expanders to try to test that with a single card.

Great explanation for beginners, well put out. It almost should be a sticky somewhere for them.

 

[voodooFX]

Yes sorry guys I posted some wrong information, my bad