Looking for advice on a case for SSDs and HDDs.

[HotFix]

Greetings.
I am working on putting together a Windows Storage Spaces server with an initial target of 8 4TB HDDs (yet to be purchased but leaning towards the HGST NAS) and 4 480GB SSDs (just picked up 4 Samsung 845DC EVOs today thank to a deal listing on these forums).

I am leaning towards the Supermicro X10SDV-LN4F motherboard as it should have plenty of horse power to run the storage and a couple of VMs, and I will put the main OS on m.2
storage leaving the disks for Storage Spaces: http://www.servethehome.com/supermicro-x10sdv-tln4f-review-platform/

I am planning on getting a 4 x 4 internal SAS port controller card, one certified to work with Windows Server 2012 R2. I haven't made up my mind which one yet. I also plan to use the motherboard's internal SATA ports for the 4 SSDs and use the controller card for the 3.5" drives.

While I'm hoping the prices of those motherboards come down a little as they are just hitting the channels now, I am turning my attention to a server case. I was originally going down the route of using a regular tower case (such as the Zalman MS800) with 2 x 3in4/3in5 3.5" drive enclosures and a 4in1 2.5" drive enclosure, but pricing that out I started hitting ~$400, and I know there are other cases out there with hard drive enclosures built in for that price.

I thought about my requirements and came up with that I wanted 12 drives (4 at 2.5") initially, but the ability to add 8 more 3.5" drives in the future w/o having to use an external case. That lead me to a case with 16 x 3.5" slots and 4 x 2.5" for my future expansion

Looking around on the forums here I started leaning towards the Norco RPC 4220 as it seemed to fit the bill. However I found threads where the 4220 doesn't fit in some racks and then I found this one where the enclosures limit the performance of SSDs: https://forums.servethehome.com/ind...ons-and-mods-on-the-new-norco-4220-case.4111/

So my question to the community is what case would you recommend for someone who wanted to populate 8 x 3.5" and 4 x 2.5" drives initially, and add 8 x 3.5" drives in the future (and why), but they didn't want to sacrifice SSD performance? I don't care either way about rack mounts because I would like to stay away from racks if I can help it, and I can lay a rack mount case on a table.

If push comes to shove I will go back to my original plan and get a generic tower case with 2 x Supermicro 5in3 3.5" enclosures and a 4in1 2.5" enclosure, and loop 2 of the 4 internal SAS controller ports external to another case when I need it in the future.

Sorry if I was rambling there, that was a bit of a brain dump. I am mostly focused on the case a the moment which is why I put this in the Chassis and Enclosures forum.

 

[j_h_o]

I've had issues with CSE-M35T-1/CSE-M35TQ in the past, FWIW -- perf issues. You'd use the 4 SSDs in a storage pool as well? Or use 1 as write cache for the Storage Pool?

Any concerns about noise from the chassis? I generally lean towards Supermicro chassis with SQ power supplies.

I like my SC846 -- but perhaps 24 bays is overkill for your purposes? Supermicro | Products | Chassis | 4U (SC847, SC848 are fine too...)

Since you're not tied to a rack, the other option is to get a smaller chassis for the system, then SAS out to a 3Gbps Rackable for spinners if you're not terribly worried about throughput: 3U SGI Rackable SE3016 16 Caddies 3 5" SAS SATA Storage Expander JBOD NAS San | eBay

 

[T_Minus]

SM 846 is very common and affordable. IMHO, it's A TON better than any NORCO or other cheap case. When I first started getting rack mount gear I got some nORCO and other generic cases, and was pissed at them.... finally got my 846 a while ago and I was converted/convinced. I now have almost exclusive SM chassis.

$300-400 to your door.

$25-35 1200w GOLD 80+ PSU

Enough room to mount SSD inside if you wanted, and even a "legit" mount option.

I have the SGI unit from them, and don't expect something as sexy as the picture. Mine looks like it was dragged through a data center with a dirt floor, and then dropped a couple times. I haven't used mine yet but I've heard they're loud, and I only planned to use spinning disks.

 

[HotFix]

I've had issues with CSE-M35T-1/CSE-M35TQ in the past, FWIW -- perf issues. You'd use the 4 SSDs in a storage pool as well? Or use 1 as write cache for the Storage Pool?

Any concerns about noise from the chassis? I generally lean towards Supermicro chassis with SQ power supplies.

I like my SC846 -- but perhaps 24 bays is overkill for your purposes? Supermicro | Products | Chassis | 4U (SC847, SC848 are fine too...)

Since you're not tied to a rack, the other option is to get a smaller chassis for the system, then SAS out to a 3Gbps Rackable for spinners if you're not terribly worried about throughput: 3U SGI Rackable SE3016 16 Caddies 3 5" SAS SATA Storage Expander JBOD NAS San | eBay

Rats on the performance issues from the CSE-M35TQ - I had high hopes on that since it hat the 92mm fan and supported SES-2.

I am planning on using the SSDs as tiered storage for the pool, including allocating some of their space to write-back cache. I have never set one of these up before so I may not have all of my terminology right, but I am doing as much research as possible (with both internal and external resources).

Noise is a major point of concern, since this will be sitting in my office with me. One of the pluses with the Norco case is I could get the 120mm fan wall and use some lower noise 120mm fans.

I looked at the SC846, but the cheapest I could find was the bottom end one with dual 900 watt power supplies for $900:
SUPERMICRO SuperChassis CSE-846TQ-R900B Black 4U Rackmount Server Case 900W Redundant - Newegg.com

I'm not married to the rack idea, so an external enclosure would be fine as long as it performed adequately. I don't have the SAS HBA yet, so I could go internal or external. I do want it to perform at 6G speeds though to take full advantage of the SSDs. That link to the Rackable solution is a neat idea.

SM 846 is very common and affordable. IMHO, it's A TON better than any NORCO or other cheap case. When I first started getting rack mount gear I got some nORCO and other generic cases, and was pissed at them.... finally got my 846 a while ago and I was converted/convinced. I now have almost exclusive SM chassis.

$300-400 to your door.

$25-35 1200w GOLD 80+ PSU

Enough room to mount SSD inside if you wanted, and even a "legit" mount option.

I have the SGI unit from them, and don't expect something as sexy as the picture. Mine looks like it was dragged through a data center with a dirt floor, and then dropped a couple times. I haven't used mine yet but I've heard they're loud, and I only planned to use spinning disks.

Where are you finding the SC846 for $300-400? Or are you looking at used options on Ebay? If so I found this one for $199:
Supermicro 846E1 R900B Barebone Server Chassis with 24x Trays | eBay
My big issue with Supermicro cases is the 15 different versions they have and trying to figure out what the subtle differences are. Like with the one I linked, I had to dig on Supermicro's site to find it was an older discontinued 3g variant.

I wouldn't mind a redundant PSU but I don't *need* one. A Gold PSU sounds good - I think someone told me the higher the rating the more energy efficient they are and the better the noise level. Is that a fair assessment?

Are you addressing the SGI unit comment to j_h_o? If not you lost me.

Either way thank you both for responding!!!

 

[markarr]

Yes looking at the SC846,
846E1x is SAS1/3G
846E2x is SAS2/6G
846TQ is SAS/SATA non expander with individual sata ports on the backplane
846A/ 846BA is SAS/SATA non expander with mini sas ports on the backplane

The part after the dash is the power supply information

 

[T_Minus]

There are also ebay sellers who replace the sas1 backplane with a sas2 as well, so don't rule out based on model # alone.

Supermicro SC846E16 R1200B X8DTE F BAREBONES 24 Bay 4U Server | eBay

Tell them you don't want the motherboard, or raid cards, but you want rails, and you'd pay $375 to your door, and see what they say. I got 2 of mine from them. (Verify it's SAS2)


And, you are correct. I was replying about the SGI unit mentioned too.

 

[markarr]

Good to know about some sellers swapping backplanes, it would be possible to go the otherway EL2 being replaced with EL1 as well if they had one with a bad backplane.

 

[T_Minus]

Yeah, I've had CERTIFIED swap out for me, and have paid them anywhere from 200-460 delivered for the 846 either empty or with psus & mobos & ram (1366),etc... all the 846 I have have the SAS2 backplane too. I also got a damaged unit from CERTIFIED but they took care of it, and after some hammering I actually racked it yesterday

 

[HotFix]

You guys are awesome.

Yes a 24 bay enclosure is a little overkill for the 20 drives I am thinking worst case scenario, but I'm sure I can make that work and it sounds like a better idea than the Norco.
Two questions on the 846 variants:
1. How noisy are they and can the be quieted?
2. Are there any SATA 6G SSD throughput concerns like with some of the 5in3 enclosures?

 

[markarr]

You guys are awesome.

Yes a 24 bay enclosure is a little overkill for the 20 drives I am thinking worst case scenario, but I'm sure I can make that work and it sounds like a better idea than the Norco.
Two questions on the 846 variants:
1. How noisy are they and can the be quieted?
2. Are there any SATA 6G SSD throughput concerns like with some of the 5in3 enclosures?

For me I plugged the fans into the mb and it quieted it down pretty good. The psu can be more annoying than the case fans, the non gold/platinum ones can be quite annoying.

Shouldn't have issues with the supermicro chassis with the EL2 expander.

 

[RyC]

I have an 846 and the non gold/platinum PSUs are going to be unbearable if it's going to be in your office. I replaced them with a platinum PSU and the noise went down dramatically, but you're still going to be able to hear the system

 

[T_Minus]

The platinum are noticeable still, and the fans can be loud or hardly noticeable depending on your office temperature. The fans should be PWM and managed by the motherboard so they go nearly silent to near jet engine

I think you could modify an 846 very easily to be much more quiet, but it will take some time getting it apart, re-sizing some holes, mounting the new fans, etc...

 

[HotFix]

Ok, another dumb question in probably a line of dumb questions about the SC846. I was reading up on it in the manual here:
http://www.supermicro.com/manuals/chassis/tower/SC846.pdf
After looking at one of the backplanes for it in this post:
Supermicro Server 4U 24 Bay SAS SATA Backplane SAS846EL1 | eBay

It looks like the old SAS846EL1 had only 3 SAS connectors on it because it was not dual channel. That means to me each SAS channel would handle 8 drives correct? If so then why am I seeing SAS cards that appear to handle only 4 drives per channel like this new one from Adaptec:
Adaptec by PMC | Adaptec Series 7H HBAs
Model 71605H
I want a HBA that is certified with Windows Server 2012 R2, and I am pretty sure how I got to the Adaptec one.

If the SAS version of the case requires a controller that can handle 8 drives per channel, then should I consider the SATA variant which I believes requires a special cable with 4 SATA breakouts + the SES signal cable (I think it's called a sideband cable)? Or am I am just being dumb and the fact that I plan on using SATA drives (SSD and HDD) means I can't use a backplane with SAS connectors?

Sorry for asking so many questions, normally I work on ProLiant servers where all of this level of detail is already dealt with.

 

[T_Minus]

It's an expander backplane. So 1 port can do all 24 drives.

 

[T_Minus]

"The SuperMicro backplane also gives us the ability without any additional controllers to daisy chain to additional external chassis using the built in expander. This simplifies expansion significantly, allowing us to simply add a cable to the back of the chassis for additional enclosure expansion."

RE: Chassis Selection » ZFS Build

He has pictures and more explaining, maybe give that a look

 

[HotFix]

"The SuperMicro backplane also gives us the ability without any additional controllers to daisy chain to additional external chassis using the built in expander. This simplifies expansion significantly, allowing us to simply add a cable to the back of the chassis for additional enclosure expansion."

RE: Chassis Selection » ZFS Build

He has pictures and more explaining, maybe give that a look

That definitely helped. I think what I was missing is even though the Adaptec card has 4 x 4 ports on it, it has "Support for SAS expanders" which I believe means I could just use a single cable and still get 16 drives. Is that right?

Follow on question - when you use the SAS expander of 24 drives, but use a HBA that only supports 16 drives, can also use a SAS reverse breakout cable to leverage 4 SATA ports on the motherboard? Or is that just plain crazy talk because the two would fight each other? Even with that approach I wouldn't be able to use 4 of the drives.

 

[T_Minus]

Your HBA limits the maximum # of devices total, check that #. That's all you're concerned about because you're not using the HBA direct to fan-out cables you're using it with an expander or more than one daisy-chained.

71605H: 4 internal mini SAS HD (SFF-8643)

The reason it says 16 limit is because that's 4 drives per fan-out cable * 4 connections. That is NOT the HBA maximum drive limit but the max you can do from fan out without an expander.

You can use onboard, HBA, and RAID cards all at once on 1 motherboard it just depends how you configure them and use them. For example I'm using a M1015 HBA for my 24 hot swaps, then an Intel RAID card or 2 for external enclosures, and possibly the on-board RAID controller for mirrored SSD.

 

[HotFix]

Your HBA limits the maximum # of devices total, check that #. That's all you're concerned about because you're not using the HBA direct to fan-out cables you're using it with an expander or more than one daisy-chained.

71605H: 4 internal mini SAS HD (SFF-8643)

The reason it says 16 limit is because that's 4 drives per fan-out cable * 4 connections. That is NOT the HBA maximum drive limit but the max you can do from fan out without an expander.

You can use onboard, HBA, and RAID cards all at once on 1 motherboard it just depends how you configure them and use them. For example I'm using a M1015 HBA for my 24 hot swaps, then an Intel RAID card or 2 for external enclosures, and possibly the on-board RAID controller for mirrored SSD.

Thank you for your patience and great explanations. I feel like I am learning at a very quick pace part of server configurations I have never had to deep dive before.

I checked the Adaptec cards and they all say they support 256 drives, even the 2 port HBAs, through an expander. So that part is now clear. It doesn't sound like I will need to use the SATA ports on the motherboard at all, which simplifies the configuration greatly.

I then went and looked at the SC846 manual and saw the various ways you could connect a HBA to the storage. I saw one way was to connect 2 channels from an HBA to PRI_J0 (the port all the way right as you look at the board) and to SEC_J0 (the 4th port from the right). They make the comment of "The SAS-846EL2 backplane has two expanders which allow effective failover", so would you only do this if you wanted failover, of will the HBA take advantage of the two paths/channels to the enclosure to double the effective bandwidth to the drives? If so does it matter if they are SATA which are single channel only?
I hope that made sense because it sounds like I could get away with a 2 port HBA versus a more expensive 4 port HBA, and I might still only be able to use 1 of the channels from the card. I thought I head read somewhere that you could saturate a SAS channel and that you should use multiple channels from a HBA to an enclosure to provide more bandwidth, I'm just not sure if that is what happens with the SC846 especially when used with SATA drives.

Thanks again for your help (and the others as well).

 

[T_Minus]

Glad to help. Like you this is new to me in the last 15 months so it's somewhat fresh in my mind, glad it's helping you make your decisions. You're a step ahead of me I spent thousands of dollars and testing to learn, and figure things out These forums save time and money!!

You're starting to ask questions slightly above my experience, but here's what I believe the answers to be based on my research.

- For Fail-Over / Redundancy you could use a 2nd raid card on that other port so if the 1st fails the second will be utilized. How you configure will be unique to your OS and File System, but I'm not sure how to configure it at all as I've never done this.

- I don't think SATA are double channel so only SAS will benefit from that if you have it setup/configured like that. I don't believe you'll see 2x bandwidth on the same backplane with 2 raid cards, a single card with SAS drives will still > than SATA.

- You don't want to mix SATA and SAS drives on the same backPlane from what I've gathered.

Someone else will have more detailed info on these questions/answers than me.

But, for home use 1 raid card to your expander with SATA drives is sufficient.

 

[HotFix]

I've been doing more research and think I can articulate my concern/thoughts a little more clearly now.

I was thinking about using a single HBA with 2x connections to the expander to double the "wide" bandwidth from 24Gbps (4 * 6 Gbps) to 48Gbps. The reason I was thinking about this approach is that from what I read the 4 high performing SSDs I have could almost saturate 4 * 6Gpbs connections and not leave much room for the throughput to the 8 * HDDs I wanted to use as well.

It looks like I would need a "interposer" to get the SATA drives visible over 2 SAS connections, and it doesn't look like there are any integrated with the SC846 expanders nor is there any room in the drive sleds with the 3.5" HDDs. So that option appears to be out.

So unless I am missing something, I would need to not user an expander and instead use the SAS connections to individual 4 * 6Gbps channels to 4 drives each. This is the only way with SATA drives to ensure I am not being throttled/rate limited by the available bandwidth to the expander over a single SAS connection.

Does that make sense? Am I smoking crack on some of my assumptions?