Looking for advice

[MrBucket101]

Hey guys,

first post, so I'm going to make it a doozie! Hoping this is in the right section!

I'm looking into building an ESXi rig, and create a virtual storage server. Reason I'm looking into ESXi, is because I think it would mesh well and provide some nice flexibility, as well as just be a fun project. My current "server" is sort of a mess. I installed a GPU/sound card and so it now also functions as my HTPC. I'm currently running Ubuntu 11.10, and I'm using mdadm to create a 4TB software RAID5 using 5 - 1TB drives. I'm also running a couple headless VMs from this machine as well, which is what initially attracted me towards ESXi.

Not sure if this is important or not, but I'm using samba currently to share my data throughout my network. My goal is to use my main gaming rig to function as my new HTPC, and stream movies and TV shows from the new machine I'm building.

Due to the decreasing price of 1TB hard drives, I thought it would be a good idea to move to a dedicated server only setup, and utilize many 1TB drives, that way I would be able to simply expand my array and keep going. I understand if I get a RAID controller, and decide to use hardware RAID, then I will need to rebuild the array.

I currently have this motherboard and this CPU

I'm looking into buying a Norco 4224.

Here is where I am stuck. I know in order to access all 24 drive bays I will need to buy a controller of some sort.

One of my friends suggested I buy this motherboard and then purchase an SAS expander. The motherboard has a built in SAS controller and he told me that SAS expansion cards are much cheaper than a RAID controller. The overall idea here was to save money. Though I'm a little hesitant to purchase a new motherboard simply to use its onboard SAS controller.

Basically, I am just looking for a cost efficient way to gain access to all 24 drive bays. Speed isn't really a concern of mine, the disks will primarily just be used for storage and streaming HD content throughout my house. That's pretty much the only requirement.

I haven't decided on hardware or software raid either, each has their ups and downs in comparison to cost vs performance. The main reason I went with software RAID initially was due to cost (FREE!!!), but if I need to buy a RAID controller to gain access to the additional drives. Wouldn't it then make sense to use the hardware RAID built into the card???

Any advice on setup, configuration options, or even flat out opinions on how everything sounds, would be greatly appreciated. Don't hold back!!!


Thanks!!!

 

[RimBlock]

The motherboard looks fine. Dual ESXi compativle lan, multiple PCIe which you can have in 8x, 8x, 4x, 4x config. Most of the controllers are likely to be in x8 config at the lower to mid level.

The CPU is good. E3-1220 v1. 4 cores, 4 threads.

The Chassis you suggest is very popular. I prefer Supermicro on quality grounds but you pay a heafty price for it.

Most entry / mid level raid cards support 8 drives without any need for an expander. You will find that LSI, a core manufacturer of raid cards, provides other manufacturers with OEM versions (IBM / Intel being two notables). Some of the OEM cards can be flashed back to the LSI equivalent version.

One of the most common entry level cards is the IBM m1015. It allows you to connect up to 8 drives natively, supports expanders and is a rebadged LSI 9220. It can be flashed to a LSI 9211 or 9240. There are usually two firmware options you can flash with and they are IT or IR. IT makes the card act like a dumb controller and IR utilizes the cards raid ability (if any). There cards support up to raid 0, 1, 10, 0+1 depending on which firmware you flash the card with. Do note, however, that it can be hard to find a motherboard that will allow you to flash the card. The Intel RS2WC080 is also a rebadged LSi 9220. They appear on EBay and Amazon as people buy servers with them and then upgrade to a better card and sell them off or old servers are being retired and broken down for sale. These cards are pretty much rock solid. They can support raid 5 with a feature key but with no cache and BBU they are slow and you run a higher risk of data corruption.

Taking a step up is the LSI 9260 which can be found as the IBM M5014 / 5015 (256 & 512MB cache ram receptively). These allow raid 5 with their cache but you would be advised to add a BBU (battery backup unit) in order to protect against data corruption if you have power issues. The RS2BL080 is the Intel equivalent.

Taking a step sideways you also have the HP P410 which is also well used with ESXi and allows the addition of cache (generally comes with 256MB) and a BBU or FBWC (Flash Backed Write Cache) which does not need a BBU. The P410 can take up to 1GB FBWC. Its big brother, the P812, comes with 1GB FBWC (upgradable to 4GB) and connections for 8 internal drives and 16 external. You may be able to find them around the US$600 mark if you are lucky. You can always route the external connections back to the internal bays if needed.

The new generation of controllers are now coming out with models like the LSI 9266 and the P420. These have better specs with the new bells and whistles but they command that new tech premium.

Someone has been spotted selling M5014 controllers on EBay for US$130 inc BBU which is a great deal although he BBU units may need replacing soon at around US$100 a pop.

Expanders allow the connection of multiple drives to limited controller connections. They are a cheap way of adding more drives and act like switches for traffic going back and forth between multiple drives and the controller (without an expander you would usually have just one connection to each drive). There are two popular ones that are quite common in white box setups. The HP Expander, which works very well with the HP controllers, and the Intel RES2SV240. The HP expander requires a PCIe slot for power but the Intel one does not making it very popular. Be aware that you are still limited by the connections between the controller, drives and PCIe bus speed limits. For hdds that is not really an issue but it can impact arrays of multiple SSDs.

Connections in your case are usually by way of a SFF-8087 socket on the controller / expander and the Norcos backplane. Each connector can support 4 channels (usually used for 4 drives in controller to backplane configurations but more like 4 bonded network cables in situations with an expander). There are also SFF-8087 (controller) -> SATA (Drives) breakout cables for connecting controllers to individual drives (forward breakout) and SFF-8087 (backplane) -> 4x SATA (Motherboard) for connecting a backplane to a motherboards SATA ports (reverse breakout). The forward or reverse are not interchangeable.

Ok, after that primer....

Go for the $130 M5014, be prepared to shell out for another battery in the next 12 months and as you expand, add a SAS expander. Check EBay every now and then as they do pop up sometimes.

RB.