Suggestions for my first real server

legen

Active Member
Mar 6, 2013
208
35
28
Sweden
The E5 system I built had a $1000 budget as well. I ended up with an ASUS Z9PE-D16, 1x E5-2620, and 24GB of ram. $1000 won't go very far into an E5 build compared to the Dell deal.
Yeah you really get less per $ with the E5 alternative, however going with the E5 gives great room for later upgrades.

I think your choice depends on what you really want to end up with. They are really different outcomes...

If you want a "practical" server to run "home production" kinds of VMs (simple web server, SIP-PBX, TV tuner, etc) go with the E3-based server. You can afford a faster CPU, it will run lower power, you can make it quieter, etc.

If you want a "real" lab and want to play with VM alternatives (ESXi vs KVM) and want maximum capability per $ (or Euro) go with the C6100. However, do understand that this will end up with a system that is power hungry, hot and loud. Without mods it is not very friendly in a home/apartment environment. Don't forget its also one to two generations behind in technology - its cheap because its getting a bit old. But it is the best deal on the market right now for a small server cluster.

If you want something to play with higher-power solutions go with the E5 (though I'd advise waiting a few months for IB). Just understand that this is probably a choice focused on future upgrades and growth. Unless you are made of money you'll have to settle for a slower CPU than you get for the some money in the E3 lineup.
Your first alternative is quite similar to what i have today (my whitebox), and an E3 upgrade is not that big a difference im afraid.

The real lab sounds tempting. I thought about going with 2x nodes running for my VMs and ZFS, this should only be ~240 W (my whitebox todays does 160W).
I'm also going to remove the 20000 RPM fans like you did with yours. I think doing this should make it possible to run it in a bigger closet i have in my current apartment.
Me and my partner are also planing to move now that we are both graduates and getting a dedicated server space is on the list of priorities :)!

I'm actually looking into the E5 alternative since it gives me room to expand later on, more memory and maybe Ivy cpus.

I have been looking at the C6100's and other cheap options for my home lab as I have outgrown my 32GB, E3 1230V2 box, and thought I would throw in an AMD build :)

Asus KGPE-D16 motherboard - dual socket G34 with 16 dimm slots (ASUS - KGPE-D16)
They seem to go for $350 to $450 depending on if you go new or used

2 x AMD opteron 6128 - 8 core processors - $30-50 each on ebay
16 x 4GB SDRAM - $450 or so brand new, but I am sure you can find cheaper on ebay

RAM is limited to 256GB RDIMM or 64GB with standard UDIMM's
CPU's can be upgraded right through to the latest Opteron 6386SE which is a 16 core 140W monster

The good thing for you is the shipping costs will be low as these are all small packages, and then you can pick up most of the parts locally.
The motherboard is SSIEEB which is EATX size so it will fit in a lot of mid/full tower cases.
PSU can just be a high quality unit like Seasonic etc.

Thought I would throw it out there for consideration :)
I googled around and checked benchmarks this morning. However the Opteron family just seems to be too power hungry and deliver poor performance in comparison with the E5 alternative. To some extent i feel that i can spend the extra bucks to get the better intel performance.

Other
Is it possible to use a raid-card (M1015 should fit) in one C6100 node and connect it to the 3.5'' hotswap enclosures? For instance use 6x3.5'' slots for one node? I tried looking at the pictures and it looks like those ordinary sata cables going to the hdd-enclosure could be wired to go to a raid-card instead.

Thanks!
 

Dragon

Banned
Feb 12, 2013
77
0
0
I run an Opteron 4170HE in a Supermicro board, H8scm-f with 12 hard drives powered by a Seasonic gold X-560 PS. PS fan never runs, not enough power being drawn by the system. My APC 1500 UPS, on its own, draws more power than this server running 6-8 VMs. I looked at Intels offerings and came to the conclusion that the Opteron 4XXX series was more than what I needed, not more than what I would like though. ESXi is what makes my server sing and it brings out what's best in the hardware.
I too built this server with long term power use in mind.
Good for you, for me the Opterons only become a problem under high load, when a single chip system draw more than 250w.

But then again, the $200 4170 HE (brand new) has the same performance as the $45 G550 (brand new), if ECC is required, pay $20 more and you get the faster G2020 with ECC.

AMD needs to wake up.
 

zer0sum

Active Member
Mar 8, 2013
389
144
43
Good for you, for me the Opterons only become a problem under high load, when a single chip system draw more than 250w.

But then again, the $200 4170 HE (brand new) has the same performance as the $45 G550 (brand new), if ECC is required, pay $20 more and you get the faster G2020 with ECC.

AMD needs to wake up.
Same performance in what exactly?
 

Dragon

Banned
Feb 12, 2013
77
0
0
Same performance in what exactly?
In overall performance. 4170HE has more throughput but G550 does each process a lot faster.

In many real world cases the 4170HE is actually much slower than the G550.

http://www.spec.org/results_search.html
AMD Opteron 4170 HE
SPECint2006 Integer score: 19.9
SPECfp2006 Floating point score: 27.5
SPECInt_rate2006 Integer throughput: 98.3
SPECfp_rate2006 Floating point throughput: 80.2

Intel Celeron G550
SPECint2006 Integer score: 32.6
SPECfp2006 Floating point score: 37.5
SPECInt_rate2006 Integer throughput: 58.1
SPECfp_rate2006 Floating point throughput: 56.6

AMD Opteron 4170 HE
CINT2006 Result: IBM Corporation IBM System x3250 M4 (Intel Celeron G550, 2.60 GHz)
Intel Celeron G550
CINT2006 Result: Tyan Tyan YR190B8228, AMD Opteron 4170 HE (test sponsored by Advanced Micro Devices)

On perlbench the Opteron 4170HE is ~50% slower
On bzip2 the Opteron 4170HE is ~25% slower
On h264ref the Opteron 4170HE is ~90% slower

The 4170HE has more cache so it's going help out in some unique situations, but then again the $60 G2020 has even better performance than the G550 and comes with ECC support, plus the power savings being 22nm.

I'd like to use my cash to support the underdog (in this case, AMD), but I don't like my power company enough to do so, not today.
 
Last edited:

RimBlock

Member
Sep 18, 2011
788
8
18
Singapore
The 4170HE has more cache so it's going help out in some unique situations, but then again the $60 G2020 has even better performance than the G550 and comes with ECC support.
Why do you believe the G550 doe not support ECC ram ?.

Whilst I have not tested all desktop processors, since Sandy Bridge ECC support has been part of the PCH (motherboard chipset).

Looking at the datasheet for the 2nd gen core processor family, under memory support (page 11) is this comment...
The type of memory supported by the processor is dependent on the PCH SKU in the target platform
— Desktop PCH platforms support non-ECC un-buffered DIMMs only
No comment is made of server PCHs (deliberately IMO). The C202/204/206 support ECC ram and some of the 2nd/3rd gen core processors (Bios depending).

I also have a client running ZFS off a Intel S1200KP with a G620 and ECC ram. I have also compared an E3-1230 and an i3-2100 on a Supermicro X9SCM-F board (C204 PCH) with ECC ram and the ram info in the BIOS was the same for both processors.

Having said all that I have not tested the G550 on a C202/204/206 board with ECC ram so if you know it does not work I would be very interested to hear so I can take a closer look and possibly get a G550 to test for myself and verify.

Interestingly, Intel have seemed to have removed references to being able to use desktop processors on their S1200XX range of boards.

RB
 

Dragon

Banned
Feb 12, 2013
77
0
0
Why do you believe the G550 doe not support ECC ram ?.
Because the G550 doesn't show up on this list:
ark.intel.com/search/advanced?ECCMemory=true

since Sandy Bridge ECC support has been part of the PCH (motherboard chipset).

I also have a client running ZFS off a Intel S1200KP with a G620 and ECC ram. I have also compared an E3-1230 and an i3-2100 on a Supermicro X9SCM-F board (C204 PCH) with ECC ram and the ram info in the BIOS was the same for both processors.
Yes, but at the same time Intel has been deliberately crippling their desktop chips to save their server department. ECC isn't some new and expensive technology, AMD had it for years in their desktop chips, it's all business decisions on Intel's part, I haven't gone through the list but every desktop chip I looked at came with some kind of calculated limitation to allow the Xeon equivalent to exist.

If G620 supports ECC then Intel is lying about its specs and so none of the hundreds of its other non-ECC chip specs can be trusted, but who's going to test them all? A 22nm/2.9GHz G2020 is only $60, why bother testing the $40 32nm/2.6GHz G550.

Now if there is a Xeon E3-1290v1/v2 desktop equivalent (i5-2550K,i5-3570/K,i7-3770/K etc) that supports ECC and saves me the $500 Xeon tax, I would be very interested.
 
Last edited:

cactus

Moderator
Jan 25, 2011
829
77
28
CA
With the Pentium G6x0 there is no reference to it not supporting ECC like say an i5. Dragon, you are right in saying it is not listed as True and so it would not come up in your search. I have a G630 and it has been run with ECC ram exclusively on two different motherboards, but they are both server/workstation PCH(C204, C206). Now with that said, I can not definitively say if it is using the ECC functionality, except that I set it in BIOS to be on. I tried to test with MemTest, but it was soon after sandy bridge came out.

Also, with the AMD 4170HE you get some things that you don't on the equal performance Intel side. The AMD gives you more memory throughput, higher memory cap(128GB vs 32GB), more PCI-e throughput and the option of SMP giving you even more memory throughput and a 256GB memory cap. So sure from a purely compute frame of reference, the 4170HE is a silly product, but that is only one use.

So it all comes down to what is the machine's role. If you need IO and lots of memory the 4170HE is worth the extra cost.
 

RimBlock

Member
Sep 18, 2011
788
8
18
Singapore
Because the G550 doesn't show up on this list:
ark.intel.com/search/advanced?ECCMemory=true
Neither does the i3-2100 I personally verified and the G620 my client is using.

Yes, but at the same time Intel has been deliberately crippling their desktop chips to save their server department. ECC isn't some new and expensive technology, AMD had it for years in their desktop chips, it's all business decisions on Intel's part, I haven't gone through the list but every desktop chip I looked at came with some kind of calculated limitation to allow the Xeon equivalent to exist.
Sure, it is all marketing. The 2nd gen i3 specs on Ark stated it was ECC compatible. Intel took that out later.

If G620 supports ECC then Intel is lying about its specs and so none of the hundreds of its other non-ECC chip specs can be trusted, but who's going to test them all? A 22nm/2.9GHz G2020 is only $60, why bother testing the $40 32nm/2.6GHz G550.
If the G550 is cheaper and does the job then why not consider. If a buyer wants the latest tech and is happy to pay a bit more then there is no issue there either. If someone pays more thinking there is no cheaper alternative then that is not so good. Intel is not doing anything but muddying the waters.

Now if there is a Xeon E3-1290v1/v2 desktop equivalent (i5-2550K,i5-3570/K,i7-3770/K etc) that supports ECC and saves me the $500 Xeon tax, I would be very interested.
The i5 range are locked out of the server / workstation PCHs that enable ECC support. This I have also tested by trying an i5-2400 in a C204 board (by mistake, though it was an i3-2120 and so was happy and sad at the same time ;) ).

The i7 range I have not played around with but would expect the same may be true.

Edit: There is only a US$100 difference between the E3-1270v2 (4/8 & 3.4GHz) vs the i5-3570K (4/4 & 3.4GHz). Above that E3 pricing does jump wildly but for not much gain (200 points on CPUMark between the E3-1270v2 and the E3-1290v2).

RB
 
Last edited:

RimBlock

Member
Sep 18, 2011
788
8
18
Singapore
With the Pentium G6x0 there is no reference to it not supporting ECC like say an i5. Dragon, you are right in saying it is not listed as True and so it would not come up in your search. I have a G630 and it has been run with ECC ram exclusively on two different motherboards, but they are both server/workstation PCH(C204, C206). Now with that said, I can not definitively say if it is using the ECC functionality, except that I set it in BIOS to be on. I tried to test with MemTest, but it was soon after sandy bridge came out.
This is also what my client say. Bios reporting ECC ram, Memtest not being 100% clear but all indications are that it was working with the ECC ram. i have not come across a definitive way of being able to test the ECC ram is being used in ECC mode (although I have not really been looking hard). The documentation concerning the PCH controlling ECC compatibility or not seems to indicate it is though.

RB
 

Dragon

Banned
Feb 12, 2013
77
0
0
If the G550 is cheaper and does the job then why not consider. If a buyer wants the latest tech and is happy to pay a bit more then there is no issue there either. If someone pays more thinking there is no cheaper alternative then that is not so good. Intel is not doing anything but muddying the waters.
Take the $20 you saved buying the 32nm G550 instead of the 22nm G2020, divide it by the number of month you'll be using it (say, 36 months), the $0.55 you save per month is about the same as the extra money you'll be paying your power company.

Buy older hardware so I can rip $20 off Intel's pocket and pass it to my power company? I don't think so. :rolleyes:
 

legen

Active Member
Mar 6, 2013
208
35
28
Sweden
So i got a reply with the shipping cost of a 4 node unit, 324$ which is not that bad.

I think i might buy this unit, i just need to ask some additional questions in the C6100 thread.
 

korban

Member
Feb 12, 2013
33
1
8
Belgium
www.befr.ebay.be
Holy shit, are you sure? I am on the other side of earth and the shipping fee is still only $300.
Hi everyone,

I'v just recieve my two c6100 from Dell PowerEdge C6100 XS23 TY3 Server 8x L5520 Quad Core CPU's 96GB Mem 4X Trays | eBay

With a delivery to Belgium I'v paid 375$ for the two nodes, will be aproximatively the same to Sweden. In Belgium we have 21% taxes then I added 475$ + 10€ for duties, but this server is owsome and open so many scenarios experimentations. I belive the taxes in Sweden are at 25%
I'v also at the same reselere the tray at 20$/p

Seriously go for it.

I plan to transform 2 dual cpu nodes to mono cpu + downclock to build a pfsense cluster
2 nodes for mirored ZFS san
4 nodes with Esxi 5.1 or proxmox or openstack or ... I nedd to learn

I will do somethin like this
(sorry I don't find how to resize the picture with the IMG tag :-( )

The fact ... it's loudy if you use the 4 nodes with full spindel hdd. A SDD could be a good investment.

Hop it will help you!
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,966
4,931
113
Keep us posted on your build korban! Should make a thread on it!
 

legen

Active Member
Mar 6, 2013
208
35
28
Sweden
Hi everyone,

I'v just recieve my two c6100 from Dell PowerEdge C6100 XS23 TY3 Server 8x L5520 Quad Core CPU's 96GB Mem 4X Trays | eBay

With a delivery to Belgium I'v paid 375$ for the two nodes, will be aproximatively the same to Sweden. In Belgium we have 21% taxes then I added 475$ + 10€ for duties, but this server is owsome and open so many scenarios experimentations. I belive the taxes in Sweden are at 25%
I'v also at the same reselere the tray at 20$/p

Seriously go for it.

I plan to transform 2 dual cpu nodes to mono cpu + downclock to build a pfsense cluster
2 nodes for mirored ZFS san
4 nodes with Esxi 5.1 or proxmox or openstack or ... I nedd to learn

I will do somethin like this ....

The fact ... it's loudy if you use the 4 nodes with full spindel hdd. A SDD could be a good investment.

Hop it will help you!
Just as a side-note, here in Sweden we have 25% tax on item value + shipping (odd to include shipping but thats how it is), however its 0% for duties for computers.
 
Last edited:

RimBlock

Member
Sep 18, 2011
788
8
18
Singapore
Take the $20 you saved buying the 32nm G550 instead of the 22nm G2020, divide it by the number of month you'll be using it (say, 36 months), the $0.55 you save per month is about the same as the extra money you'll be paying your power company.

Buy older hardware so I can rip $20 off Intel's pocket and pass it to my power company? I don't think so. :rolleyes:
I believe your original point was that you would have to get a G2020 in order to us ECC ram. I believe this is incorrect based on available documentation and user reports.

Now you have moved on to initial costs and power savings which, using your own figures, would mean that you would have to run the G2020 processor for 3 years before levelling the playing field based on purchase price and power costs. An upgrade after 2 years would mean that the G550 worked out cheaper over that working period. You could then go on to start talking about the G2020 having a higher resale value etc...

Again, the point is that the G2020 is not the cheapest Intel processor that supports ECC ram. There may be many of good reasons to favor the G2020 over the G550 but that is not one of them.

Also not sure why you are concerned with ripping US$20 out of Intels pocket. I would be more concerned with ripping US$20 out of my own pocket in initial outlay when it may not be required.

RB
 

legen

Active Member
Mar 6, 2013
208
35
28
Sweden
So i finally bought a unit from PDNEIMAN on ebay.

Now only the long wait remains :)

Thanks for all the help guys!
 

Dragon

Banned
Feb 12, 2013
77
0
0
I believe your original point was that you would have to get a G2020 in order to us ECC ram. I believe this is incorrect based on available documentation and user reports.

Now you have moved on to initial costs and power savings which, using your own figures, would mean that you would have to run the G2020 processor for 3 years before levelling the playing field based on purchase price and power costs. An upgrade after 2 years would mean that the G550 worked out cheaper over that working period. You could then go on to start talking about the G2020 having a higher resale value etc...

Again, the point is that the G2020 is not the cheapest Intel processor that supports ECC ram. There may be many of good reasons to favor the G2020 over the G550 but that is not one of them.
Wtf is this? You are arguing over using the G2020 for 2 years instead of 3? Am I supposed to argue back with "But you can also use it for 4 years instead of 2"? What is the point?

Look dude, I don't know what your problem is, maybe it's the typical bitterness SG'er have against HK'er, whatever it is I tried to brush it off and end that discussion with a joke, but now you're just pissing me off.

If you have nothing better to do than nitpick then at least pay attention to what I actually posted:
"pay $20 more and you get the faster G2020 with ECC."
"the $60 G2020 has even better performance than the G550 and comes with ECC support"

The keywords you missed in both cases are "faster" and "better performance". And the reason the G550/G2020 was even bought up in this thread was because we were discussing Opteron's power consumption and cost. So no, the main point wasn't ECC and I didn't "now you move on to initial costs and power savings", power consumption and cost-performance ratio has always been the main factor, ECC was the secondary requirement. READ the actual posts, if the context already implied something then I am not going to repeat the same point over and over each and every post that follows.

Compare to the G550, the G2020 has 12% higher clock speed, 50% more cache, 34% more memory bandwidth, 15% lower TDP, with OFFICIAL ECC support, costing only $20 more, a small extra cost that you can gain back from your power saving over a few years, that is the point.

But what's so difficult about reading specs? I tried to keep my response short and simple and you're here screaming: "OMFG You're WRONG the G550 has ECC too!!!!!11!1!!1!"?

Who gives a shit about the G550, really. ECC or not, choosing the G2020 over G550 is still a no brainer, unless the whole point of this discussion is that you want a medal for knowing G550 supports ECC?

Hey genius, it's not my fault Intel hired idiots to run their specs database, if Intel decided to spend years and billions in R&D/IP/HR/Infrastructures/materials/marketing/legal/logistics, and then not tell the public about their products' true capabilities on their website, that is NOT my problem, I am not going to do their job for them over $20.

And if G2020 wasn't so much better than the G550 I might have given it more attention 2 months ago and found out some guys on [H] have already been running G550 with ECC for ages.

Like I said, if you "discovered" that the $800+ Xeons has a $300 desktop equivalent that support ECC, you'll get a medal. But "discovering" ECC on a $40 chip that runs slower and cost more to run than a $60 chip that already has ECC? Well you're not getting jack shit.

If the G550 is cheaper and does the job then why not consider
G550 is cheaper than the G2020 only if you're running a small stack in your mother's basement and you are not the one paying the electric bills.

Also not sure why you are concerned with ripping US$20 out of Intels pocket. I would be more concerned with ripping US$20 out of my own pocket in initial outlay when it may not be required.
If you can't think beyond initial costs then don't talk to me about money, you're just wasting my time.
 

RimBlock

Member
Sep 18, 2011
788
8
18
Singapore
some stuff....

Look dude, I don't know what your problem is, maybe it's the typical bitterness SG'er have against HK'er, whatever it is I tried to brush it off and end that discussion with a joke, but now you're just pissing me off.
Strange, turning it in to nationalist paranoia. Firstly I have no idea where you are from (quite frankly I really don't care) and secondly, just because I happen to be in Singapore does not mean I am Singaporean.

If you have nothing better to do than nitpick then at least pay attention to what I actually posted:
"pay $20 more and you get the faster G2020 with ECC."
"the $60 G2020 has even better performance than the G550 and comes with ECC support"
No, what you said was
"But then again, the $200 4170 HE (brand new) has the same performance as the $45 G550 (brand new), if ECC is required, pay $20 more and you get the faster G2020 with ECC."

I have highlighted the key point.

You also said in your next post
"The 4170HE has more cache so it's going help out in some unique situations, but then again the $60 G2020 has even better performance than the G550 and comes with ECC support, plus the power savings being 22nm."

My only point was to try and find out why you though the G550 didn't. I was genuinely interested as I believed it did but had not tested it personally. I though I made it clear in post #25.

I have already stated that the G2020 may be better and pointed pointed out that it is not the cheapest option if you want ECC support. I have no idea why you are so passionate about this or why you seem to feel it is a personal vendetta against you.

If I had said something that was incorrect then I would rather someone picked up on it and then I can learn something new. I certainly would not take it as a personal attack unless it was clearly done in a nasty way.

some sort of rambling rant...
tl;dnr
If you wish to continue along this road then fine, personally I would rather hear more about Korbans build and have no wish to see this thread derailed anymore than it already is.

RB