BThunderW's Hardware Stuff

Currently going through a major overhaul of my home hardware. I got a few C6100's and a couple of DL180 G6's on the way (Thanks STH!)

For the last few months I've been trying to reduce the amount of hydro and heat my home rack consumes. At it's peak I was pulling about 23A (two dedicated 120V/15A circuits) on the rack + another 7A on cooling.

I've gone through a lot of hardware changes over the years. Started with Dell PowerEdge 1750/2850's. A lot of custom built 4U boxes (cause they were cheap). Gradually moved to Dell PE2950's. Once the Dell R710's started coming down in price I started selling the 2950s and started upgrading. I couldn't believe how much quieter the R710's were over the 2950s. Although, the worst server of all was the 1750 as it was a horrible screamer. Could hear the fans pretty much throughout the house. Wife was glad when that server was gone.

I typically upgrade year round whenever I find deals on hardware. Recently I ran into some C6100's on Kijiji, way overpriced, asking was $2200, but I started digging around which led me to this site. I'm hoping that by the time I'm done my power draw will be reduce to 1/2 of what I was running before this upgrade cycle.

That being said, here's my current config:

Core Switch (Force10 S50) / 48 Port


4x R710 (2x ESXi, 1xHyper-V 2012, 1xNexentaStor + MD1000). The R710 running Nexentastor has been modded to accept a 2.5" HDD in the DVD-Rom slot (SSD Cache drive).


Also my Norco 20 Bay case which is my Media server (8x2TB + 12x1TB drives). Currently running X3400 series Xeon hardware and NexentaStor.


Not shown is another DL160 G6 which is paired to another MD1000 acting as a backup server.

The whole rack is backed up via 2x1500 APC SmartUPS (2U each). Each UPS is powered via external 2x12V Deep Cycle batteries. During power failures the 2 battery banks provide enough power to run the rack for 2-3 hours depending on the load.


"Telco Rack"
2xCable Internet (150/10 each). Modified Watchguard x750e running pfSense 2.0. Dell 5324 switch connecting the whole house.


I'm running additional hardware at two data centers. One of them is a private Half-Rack, the other one is a shared full rack. I'll post some pics of that soon plus some pics of my home office.

 

Great pictures! I do have two questions though:
1. What are you using all of this for?
2. Are those car batteries? How does that work? I would imagine cheaper than buying more from APC.

 

I'm a software developer / consultant. I often create virtual dev environments for whatever project I'm working on. I'm also running domain server/exchange/BES etc for my home network. I also like to experiment with new software products without having to install/uninstall on my workstation.

These are not typical car batteries. Unlike starter batteries, deep cycle batteries are designed to be run down to empty without damage. Starter batteries begin to sulfate when charge is below 100%. These batteries are typically used in alternative energy projects (solar, wind, etc). They cost about $90/ea. But provide much longer up time than typical APC battery. The biggest drawback that unlike APC sealed batteries, these need to be topped off with distilled water on regular basis.

 

Tonight I'm going to google maintenance free deep cycle batteries. I don't want to derail this thread, but am interested in how that all works

 

Thanks for the replies!

iirc Fore10 had the S50 and S50n. I think the n version had management software and the ability to do IPv6? Any idea which you have?

Great data points on noise. Always a concern with basement rack

 

This is an older Force10 S50. It's running SFTOS rather then the newer FTOS and it's not upgradeable. Does not support IPv6, which is fine for me (for now). It'll still work at L2 level so that means simply no advanced L3 IPv6 functions.

It does have web management on top of standard ssh/telnet/serial. It also does L3 routing which I'm not using at this point.

 

Haha you sold me your R710s.

 

LOL. Nice.

 

Ok. So I have a plethora of hardware on the way via eBay.

2x C6100
2x DL180 G6 /w 12x300GB 15K drives
2x M1015 Cards
4x SuperMicro FAN-0074L4 Fans (only one C6100 for home)
2x Watchguard x750e firewalls (more modding)

Based on what I can see the items won't get to my US address till early next week. So I'll have to make a trip sometime next week to pick that stuff up. I can't even imagine what my brokerage fees would have been.

Shipping a single server from US was quoted at $400 to my door in Canada (and takes 2 weeks). Shipping the same server to my US address is free (and takes about 6 days), go figure.

Really can't wait to start tearing into the hardware. Anyone know if the C6100 onboard NIC supports iSCSI boot?

 

APC uses really nice B.B. batteries. 3600 watts wired up at 48V (4 batteries). Each one extends the 40 minutes to 400 minutes of runtime. stupid heavy.

I am thinking of rolling my own with these tripp lite double online 1000VA I picked up for $20 new each. I suppose I could rewire the "big mama" APC boxes to these guys @ 24volt.

Man batteries are getting expensive but I am not sure OSHA would be happy with venting batteries in a data closet.

 

Actually, If you can get away with running 48V instead of 24V it'd be a lot more preferable. You get get away with lighter gauge wire as the wires will only have to carry 1/2 the current of 24V battery bank. Allows you to run longer wires, so gives you more choices when placing the battery bank.

 

West Auctions - Auction: Medical Equipment from Fertility Center ITEM: (3) APC Ultra Battery Packs and (1) APC Smart-UPS

uses like welding wire gauge man. You can daisy chain those things more than that. Each battery is 280lbs. That rig in the auction link there is probably easily 1000lbs but when you need to run 3000VA for hours (or days at lower load) on single phase input (110V 30amp) - that will do it.

 

How much were the G6's with those drives and got a link to buy more?

 

I paid about $1100 each shipped. One of them was a one-off auction and another was a custom build from this seller: eBay My World - orangecomputers2012

LOL. That's serious overkill. I don't think anyone really needs that much runtime, unless of course you live in an area that's prone to frequent black outs. The reason I use the deep cycle batteries is because they're relatively cheap. And I typically only need about an hour of run time. I usually give hydro about 30 minutes to come back, then start shutting everything down which still takes a bit to bring down all the VMs. I use write caching on Nexentastor so I want to make sure everything is committed to disk before the juice runs out.

 

Holy moly! I wonder how much runtime for those and better, how much they cost to replace batteries. Nothing I dislike more than a beeping APC unit that needs a battery replaced.

 

Woo!



Note to self, crossing the border on a Friday just before long weekend....not a good idea.

 

Ah yes, the dreaded US UPS outlet run. Shudder. .

 

Got NexentaStor installed on the DL180 G6 today. Now running with 14x300GB 15K drives plus a few 3TB drives for backups (via connected MD1000). Quite impressed with this server. Not only that box comes with 12 bays. It has two more SATA ports on the backplane, and 6 more on the motherboard. I had to upgrade the BIOS on it because it was having trouble booting with this many drives connected. The server is very quiet too. Didn't run my SPL meter but the server can't be heard over the exhaust fan in my server room.

The DL180 came with 2xE5620 and 12x300GB drives. During bench testing the server sustained about 300W at the socket. In comparison my existing MD1000 with 14x146GB drives had a draw of 330W. The MD1000 is just a dumb disk shelf with no processing power. Very nice.

I have another server just like it that I'll be prepping for my DC rack deployment in the next few weeks.

 

Curious about this. My experience to date with the DL180 puts it in the "medium loud to loud" range. Quieter than "stock" C6100 but much louder than one with the fan mods. Too loud for sustained use even in the garage - loud enough to be heard through the insulated walls. Easily louder than my prior "loudest" item, a Juniper EX2500 10gbe switch.

Would be very interested to get SPL meter data on it.

 

Gimme a few. I have the second DL180 G6. I'll toss it in a quiet room and do a measurement.