Looking for Advice - Filling C7000 blade chassis

[Bradford]

Hello,

I scored a c7000 chassis locally at what I think is a good price at $270. It included all the fans and power supplies (that run on 110) and iLO and switches I could use, as well as 4 operational blades (BL460c G7 w/ 2xE5620 and 12GB RAM).

My primary purpose is to have a home lab again, and play around with clustering, pxe booting, etc. But in order to fund this, whatever lab equipment I get I will mine Monero with in the meantime in order to hopefully pay for itself.

What's the sweet spot in older Xeons and Opterons when it comes to hashing? The Opteron 6200 series look good, but the supply seems to have dried up and they're around $100 each now. Everything I'm seeing now doesn't seem to be profitable - newer gen blades are very expensive, and older stuff just can't keep up with the power bill (which, for me, is low at $.07 but will increase when I move in the next few months).

Maybe I'll get lucky and find a local surplus auction with a bunch of blades, but that's a long shot. Are there any other places I should look for old blades besides that and eBay?

What do you think is the best move? Again, mining is just to hopefully fund the equipment and power. Thanks in advance for any advice you have.

 

[CyberianSoldier]

Honestly, I just picked up one of these things. But the fans use ~200 watts each. at 2000 watts for just the cooling, I'm not sure this is the best for energy efficiency. I'd be curious to see what people suggest.
Great price on the chassis though!

 

[markarr]

I would suggest taking a look at this for some basic power numbers based on what you want to populate the chassis with.

HPE Power Advisor

I wouldn't look any blades older than g8 if you remotely care about power. Looks like barebone blades are ~140 on ebay for g8.

 

[Bradford]

Honestly, I just picked up one of these things. But the fans use ~200 watts each. at 2000 watts for just the cooling, I'm not sure this is the best for energy efficiency. I'd be curious to see what people suggest.
Great price on the chassis though!

That's absolutely insane! I want to look and see what kind of cooling management options there are.. Maybe turn down fan speeds. I guess that's a condition with high density - tighter spaces for air and so higher speeds necessary to cool it. My new house is going to need a soundproof room in the basement for this thing.

I would suggest taking a look at this for some basic power numbers based on what you want to populate the chassis with.

HPE Power Advisor

I wouldn't look any blades older than g8 if you remotely care about power. Looks like barebone blades are ~140 on ebay for g8.

Thanks - that's what I've determined, G8 or newer, but there are many different versions of them and I'm not sure about the sweet spot in terms of CPU price and hashrate. AFAIK it's a function of cache size, but the E5-2600 series with the same L3 cache have very different hash rates.

 

[markarr]

I would look at the bl460c g8 for the blades and if your looking for more density then the 2660 v2 and 2680 v2 are decent options. 2660 v2 are 120-150 2680 are 150-180 a piece. Yea its a function of cache and core count.

 

[Evan]

Honestly, I just picked up one of these things. But the fans use ~200 watts each. at 2000 watts for just the cooling, I'm not sure this is the best for energy efficiency. I'd be curious to see what people suggest.
Great price on the chassis though!

Err 200w total yes but not 2000w, but yes a c7000 chassis is not Low power or even power efficient unless you fill them.

16 x Fully loaded with dual e5-2690v2 and 512gb ram + 2 x HBA,.... ~7000 watts !!

 

[CyberianSoldier]

no, they use ~200 watts per fan. I have 3 nodes running and it's burning 18 Amps at 210V. The fan uses 12V and up to ~15Amps.

 

[Evan]

no, they use ~200 watts per fan. I have 3 nodes running and it's burning 18 Amps at 210V. The fan uses 12V and up to ~15Amps.

Not disputing maximum usage just saying they usually don’t use that much.
What cards d0 you have it ?
I have a system with 2 passthough 10g and 2 brocade switch, that’s a test unit so only a few blades, I will check usage but it is far less than what your suggesting I am sure.

Essentially a fully loaded and 100% utilised system (e5 v2/v3/v4) can consume the whole 32amp @ 240v.
You absolutely must use the 2400w platinum psu to allow this with 3 pdu failed, if you use the 2200w psu then you will have major cpu throttling on a fully loaded chassis in a power failure situation. Got a good number of these chassis and we have stress tested under load with power failure simulated.
(Note we run ~22-24 degrees C and good volume of airflow, hotter than that and for sure you will see much greater fan speeds and hence power consumption)

 

[Evan]

In any case no way I would be running these as a home lab ! So kind of a academic discussion.
3800 watts for 4 nodes is insane, 7000 watts for fully loaded 16 blades is ok, if you use it... not a home lab

 

[Bradford]

In any case no way I would be running these as a home lab ! So kind of a academic discussion.
3800 watts for 4 nodes is insane, 7000 watts for fully loaded 16 blades is ok, if you use it... not a home lab

I agree, overkill unless I can offset the cost by some sort of profitable mining. It looks like my best bet to cheaply fill this thing with decent hardware is to keep an eye on surplus auctions and get lucky.

 

[Bradford]

no, they use ~200 watts per fan. I have 3 nodes running and it's burning 18 Amps at 210V. The fan uses 12V and up to ~15Amps.

When I tested the system in the seller's basement, he had 2 nodes idling (trying to pxe boot) and it was pulling around 600 watts from 110V - fans weren't at full speed, etc.

 

[Bradford]

It looks like my best bet is getting BL465c or BL685c blades and getting 6200 / 6300 opterons when they get cheap.

 

[Yves]

no, they use ~200 watts per fan. I have 3 nodes running and it's burning 18 Amps at 210V. The fan uses 12V and up to ~15Amps.

Sorry I don't want to be rude or anything maybe I am also mistaken. But I run a BC C7000 as a home lab toy around environment and it currently runs 10 fans + 2 OA interconnects + 2 Flex10 interconnects + 2 1GB PassThru interconnects + 1 BL460c G7 (almost max config dual X5670 + 96GB RAM with 2 SSDs) and it says 634 Watt AC

edit: here is the fan power summary